arxiv: 2604.20443 · v1 · submitted 2026-04-22 · 💻 cs.CL · cs.AI· cs.LG

Recognition: unknown

DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories

Neemesh Yadav , Palakorn Achananuparp , Jing Jiang , Ee-Peng Lim

Authors on Pith no claims yet

Pith reviewed 2026-05-10 00:08 UTC · model grok-4.3

classification 💻 cs.CL cs.AIcs.LG

keywords Theory of MindDialogue forecastingLarge language modelsBenchmarkMental state inferenceFunctional reasoningSocial trajectory prediction

0 comments

The pith

LLMs identify mental states in dialogue but mostly fail to forecast how conversations will unfold from those states.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds DialToM, a benchmark of natural human dialogues turned into multiple-choice questions, to test two layers of Theory of Mind. Literal ToM asks models to name the mental states present; Functional ToM asks whether those states alone let a model pick the dialogue path that would actually follow. Results show most models perform well on the first task yet poorly on the second, with only Gemini 3 Pro succeeding at both, and with only weak overlap between the inferences humans and models produce.

Core claim

DialToM reveals a clear asymmetry: large language models can accurately extract mental-state profiles from dialogue turns, yet the same models (except Gemini 3 Pro) cannot reliably select the state-consistent future trajectory when given only those profiles, and the semantic content of their inferences diverges from human judgments.

What carries the argument

Prospective Diagnostic Forecasting, a multiple-choice task that supplies only a mental-state profile and asks the model to choose which of several possible dialogue continuations is consistent with those states.

If this is right

Current LLM ToM capabilities remain largely diagnostic rather than predictive.
Only a subset of frontier models can translate identified mental states into forward simulation of dialogue.
Semantic divergence between human and model inferences suggests different internal representations of social context.
The benchmark supplies a concrete yardstick for measuring whether future training methods close the literal-to-functional gap.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Training objectives that reward only next-token accuracy may never produce robust functional ToM without explicit trajectory-level supervision.
Dialogue agents that must anticipate user reactions would need separate modules or fine-tuning beyond standard instruction tuning.
If the asymmetry persists across domains, it limits the reliability of LLM-based social simulation tools such as negotiation or therapy assistants.

Load-bearing premise

The multiple-choice options and human verification process truly require models to reason from mental states rather than exploit surface patterns or dataset regularities.

What would settle it

Construct a new test set in which correct trajectory choices require genuine state reasoning while surface cues point to the wrong answer; if models still succeed at the same rate, the functional-ToM claim is falsified.

Figures

Figures reproduced from arXiv: 2604.20443 by Ee-Peng Lim, Jing Jiang, Neemesh Yadav, Palakorn Achananuparp.

**Figure 1.** Figure 1: The DialToM Benchmarking Pipeline. The workflow illustrates the transition from Literal ToM (Retrospective [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

read the original abstract

Large Language Models (LLMs) have been shown to possess Theory of Mind (ToM) abilities. However, it remains unclear whether this stems from robust reasoning or spurious correlations. We introduce DialToM, a human-verified benchmark built from natural human dialogue using a multiple-choice framework. We evaluate not only mental state prediction (Literal ToM) but also the functional utility of these states (Functional ToM) through Prospective Diagnostic Forecasting -- probing whether models can identify state-consistent dialogue trajectories solely from mental-state profiles. Our results reveal a significant reasoning asymmetry: while LLMs excel at identifying mental states, most (except for Gemini 3 Pro) fail to leverage this understanding to forecast social trajectories. Additionally, we find only weak semantic similarities between human and LLM-generated inferences. To facilitate reproducibility, the DialToM dataset and evaluation code are publicly available at https://github.com/Stealth-py/DialToM.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

DialToM shows LLMs handle literal mental-state detection in dialogue but mostly cannot use it to forecast consistent trajectories, with the multiple-choice format leaving some doubt on how cleanly it isolates functional reasoning.

read the letter

The core result is straightforward: models do well at pulling out mental states from natural dialogues but then fail to pick state-consistent future turns in most cases, except Gemini 3 Pro. The paper also reports only weak semantic overlap between human and LLM inferences about those states. DialToM is built from real conversations rather than scripted ones, with human verification on the options and a public release of the data and code. That combination of natural source material and the prospective forecasting task is the main novelty relative to earlier ToM benchmarks. It gives a concrete way to test whether state recognition actually helps with downstream social prediction. The public artifacts make it easy for others to run their own models and check the numbers. The asymmetry finding is presented clearly and the work cites the relevant prior benchmarks without obvious gaps. The main soft spot is the multiple-choice setup itself. Models could succeed or fail by matching surface features of the options to the profile text or by relying on general dialogue patterns instead of carrying out state-driven inference. Human verification ensures the choices are plausible, but it does not include the kind of controls that would rule out those shortcuts. The low semantic similarity between human and model inferences is noted but left somewhat open; it could reflect different internal representations rather than a direct flaw in the ToM claim. No load-bearing fitting or circular derivations appear in the evaluation. The paper is aimed at researchers who build or test conversational systems that need social reasoning. Anyone running ToM evaluations or dialogue models would find the benchmark and the reported gap useful to try on their own work. It is coherent on its own terms and engages the literature directly, so it deserves a serious referee even if the design questions need addressing in revision. I would send it out for peer review rather than desk reject.

Referee Report

3 major / 2 minor

Summary. The manuscript introduces DialToM, a human-verified benchmark for evaluating Theory of Mind (ToM) in large language models (LLMs) using natural dialogue data. It distinguishes Literal ToM (mental state prediction) from Functional ToM (forecasting dialogue trajectories from mental state profiles) via a multiple-choice Prospective Diagnostic Forecasting task. Key findings include strong performance on Literal ToM but poor performance on Functional ToM for most models (except Gemini 3 Pro), and weak semantic similarity between human and LLM-generated inferences. The dataset and code are released publicly.

Significance. If the reported asymmetry between Literal and Functional ToM holds, this work would be significant for highlighting limitations in LLMs' ability to apply mental state understanding to predict social interactions, with implications for conversational AI systems. The public availability of the DialToM dataset and evaluation code is a notable strength that supports reproducibility and further research in the field.

major comments (3)

[Methods (Prospective Diagnostic Forecasting)] The multiple-choice setup for forecasting state-consistent trajectories does not include controls or ablations to rule out selection based on surface features, lexical overlap with the mental-state profiles, or general dialogue priors rather than genuine state-driven inference. This is central to the asymmetry claim, as the skeptic's concern about option patterns remains unaddressed.
[Results] The exception noted for Gemini 3 Pro is presented without accompanying error analysis or qualitative comparison showing how it differs from other models in leveraging the profiles, making it difficult to confirm that its success reflects superior functional ToM rather than better artifact handling.
[Evaluation of inferences] The claim of only weak semantic similarities between human and LLM inferences requires more detail on the measurement method (e.g., embedding model used, similarity metric) and statistical significance to assess its robustness and implications for state representation differences.

minor comments (2)

[Abstract] The abstract mentions 'significant reasoning asymmetry' but does not specify the magnitude or statistical tests used; consider adding a brief quantitative summary.
[Dataset construction] Ensure that the human verification process is described with inter-annotator agreement metrics to strengthen claims of benchmark quality.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. The comments highlight important areas for strengthening the evidence behind our claims of a Literal-Functional ToM asymmetry. We have revised the manuscript to incorporate additional controls, error analysis, and methodological details as outlined below.

read point-by-point responses

Referee: [Methods (Prospective Diagnostic Forecasting)] The multiple-choice setup for forecasting state-consistent trajectories does not include controls or ablations to rule out selection based on surface features, lexical overlap with the mental-state profiles, or general dialogue priors rather than genuine state-driven inference. This is central to the asymmetry claim, as the skeptic's concern about option patterns remains unaddressed.

Authors: We agree that the absence of explicit controls leaves open the possibility that models exploit surface-level cues rather than performing state-driven inference. In the revised manuscript we have added three ablations to the Prospective Diagnostic Forecasting task: (1) a shuffled-profile control that randomly permutes the mental-state descriptions while keeping the same option set, (2) a lexical-overlap baseline that selects the trajectory option with highest unigram overlap to the profile, and (3) a no-profile control that supplies only generic dialogue priors. Results show that model accuracy drops to near-chance levels under the shuffled and lexical controls, while the original profile-conditioned setting yields the reported performance gap. These controls are now described in the Methods section and reported in a new table in Results. revision: yes
Referee: [Results] The exception noted for Gemini 3 Pro is presented without accompanying error analysis or qualitative comparison showing how it differs from other models in leveraging the profiles, making it difficult to confirm that its success reflects superior functional ToM rather than better artifact handling.

Authors: We acknowledge that simply noting Gemini 3 Pro's higher accuracy is insufficient. We have added an error-analysis subsection that categorizes failures across all models (e.g., ignoring specific mental-state cues, defaulting to high-frequency dialogue patterns). We also include qualitative examples in the appendix contrasting Gemini's correct forecasts—which explicitly reference profile elements such as “the speaker’s desire to avoid conflict”—with other models’ selections that align with surface statistics. These additions appear in the revised Results and Appendix. revision: yes
Referee: [Evaluation of inferences] The claim of only weak semantic similarities between human and LLM inferences requires more detail on the measurement method (e.g., embedding model used, similarity metric) and statistical significance to assess its robustness and implications for state representation differences.

Authors: We have expanded the relevant section to specify: (a) the embedding model (sentence-transformers/all-MiniLM-L6-v2), (b) the similarity metric (cosine similarity on mean-pooled embeddings), and (c) statistical tests (one-sample t-tests against a random-inference baseline, with reported p-values < 0.001). The weak similarity result remains robust under these details, and we now discuss its implications for divergent internal state representations between humans and LLMs. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical benchmark evaluation without derivations or self-referential reductions

full rationale

The paper introduces DialToM as a human-verified multiple-choice benchmark for Literal ToM (mental state identification) and Functional ToM (forecasting state-consistent trajectories) using natural dialogues. All claims rest on direct empirical results from evaluating LLMs on this dataset, with no equations, parameter fittings, ansatzes, or derivation chains present. The asymmetry finding and weak semantic similarity observations are reported outcomes of the evaluation protocol rather than outputs derived from prior fitted values or self-citations. The benchmark construction and human verification steps are described as independent of the model results, rendering the work self-contained against external data without any load-bearing reduction to its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only view yields no explicit free parameters, axioms, or invented entities; the work assumes standard NLP evaluation practices and human annotation reliability without further detail.

pith-pipeline@v0.9.0 · 5471 in / 972 out tokens · 33532 ms · 2026-05-10T00:08:44.170035+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

46 extracted references · 30 canonical work pages · 6 internal anchors

[1]

Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al . 2025. gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925(2025)

work page internal anchor Pith review arXiv 2025
[2]

Meta AI. 2025. Llama 4. https://ai.meta.com/blog/llama-4-multimodal- intelligence/

2025
[3]

Apperly and Stephen A

Ian A. Apperly and Stephen A. Butterfill. 2009. Do humans have two systems to track beliefs and belief-like states?Psychological Review116, 4 (2009), 953–970. doi:10.1037/a0016923

work page doi:10.1037/a0016923 2009
[4]

Mislavskii, N

Chris L. Baker, Rebecca Saxe, and Joshua B. Tenenbaum. 2009. Action under- standing as inverse planning.Cognition113, 3 (2009), 329–349. doi:10.1016/j. cognition.2009.07.005 Reinforcement learning and higher cognition

work page doi:10.1016/j 2009
[5]

We Are Explorers Together

Erika Blacksher, Charlene Nelson, Emily Van Dyke, Abigail Echo-Hawk, Deborah Bassett, and Dedra Buchwald. 2016. Conversations about Community-Based Participatory Research and Trust: “We Are Explorers Together”.Progress in Community Health Partnerships: Research, Education, and Action10, 2 (2016), 305–309. doi:10.1353/cpr.2016.0039

work page doi:10.1353/cpr.2016.0039 2016
[6]

1987.Intention, plans, and practical reason

Michael Bratman. 1987.Intention, plans, and practical reason. Stanford Univ Center for the Study

1987
[7]

Zhuang Chen, Jincenzi Wu, Jinfeng Zhou, Bosi Wen, Guanqun Bi, Gongyao Jiang, Yaru Cao, Mengting Hu, Yunghwei Lai, Zexuan Xiong, and Minlie Huang
[8]

ToMBench: Benchmarking Theory of Mind in Large Language Models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories Srikumar (Eds.). Association for Computational Linguis...

work page doi:10.18653/v1/2024.acl-long.847 2024
[9]

Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, Vincent Wade, and Benjamin R. Cowan. 2019. What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents. InProceedings of the 2019 CHI Conference on Human Factors in Computing Sy...

work page doi:10.1145/3290605.3300705 2019
[10]

Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and Others. 2025. Gemini 2.5: Pushing the Frontier with Advanced Reason- ing, Multimodality, Long Context, and Next Generation Agentic Capabilities. arXiv:2507.06261 [cs.CL] https://arxiv.org/abs/2507.06261

work page internal anchor Pith review Pith/arXiv arXiv 2025
[11]

A. P. Dawid and A. M. Skene. 1979. Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm.Applied Statistics28, 1 (1979), 20. doi:10. 2307/2346806

1979
[12]

Google DeepMind. 2025. Gemini 3. https://blog.google/innovation-and-ai/ technology/ai/google-gemini-ai/

2025
[13]

DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. arXiv:2412.19437 [cs.CL] https://arxiv.org/abs/2412.19437

work page internal anchor Pith review Pith/arXiv arXiv 2024
[14]

Daniel C. Dennett. 1978. Beliefs about beliefs [P&W, SR&B].Behavioral and Brain Sciences1, 4 (1978), 568–570. doi:10.1017/S0140525X00076664

work page doi:10.1017/s0140525x00076664 1978
[15]

Kanishk Gandhi, Jan-Philipp Fraenken, Tobias Gerstenberg, and Noah Goodman
[16]

InAdvances in Neural Information Processing Systems, A

Understanding Social Reasoning in Language Models with Language Models. InAdvances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 13518–13529. https://proceedings.neurips.cc/paper_files/paper/2023/file/ 2b9efb085d3829a2aadffab63ba206de-Paper-Datasets_a...

2023
[17]

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schel- ten, Alex Vaughan, and Others. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783

work page internal anchor Pith review Pith/arXiv arXiv 2024
[18]

Kilem Li Gwet. 2008. Computing inter-rater reliability and its variance in the presence of high agreement.Brit. J. Math. Statist. Psych.61, 1 (May 2008), 29–48. doi:10.1348/000711006x126600

work page doi:10.1348/000711006x126600 2008
[19]

P. A. Hancock, Theresa T. Kessler, Alexandra D. Kaplan, Kimberly Stowers, J. Christopher Brill, Deborah R. Billings, Kristin E. Schaefer, and James L. Szalma
[20]

Frontiers in Psychology14 (March 2023)

How and why humans trust: A meta-analysis and elaborated model. Frontiers in Psychology14 (March 2023). doi:10.3389/fpsyg.2023.1081086

work page doi:10.3389/fpsyg.2023.1081086 2023
[21]

2006.03.007

Mark K. Ho, Rebecca Saxe, and Fiery Cushman. 2022. Planning with Theory of Mind.Trends in Cognitive Sciences26, 11 (Nov. 2022), 959–971. doi:10.1016/j.tics. 2022.08.003

work page doi:10.1016/j.tics 2022
[22]

Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Bras, Gunhee Kim, Yejin Choi, and Maarten Sap. 2023. FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguisti...

work page doi:10.18653/v1/2023.emnlp-main.890 2023
[23]

Matthew Le, Y-Lan Boureau, and Maximilian Nickel. 2019. Revisiting the Evalu- ation of Theory of Mind through Question Answering. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Inter- national Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Ken- taro Inui, Jing Jiang, Vincent Ng, and Xi...

work page doi:10.18653/v1/d19-1598 2019
[24]

Mengfan Li, Xuanhua Shi, and Yang Deng. 2025. RecToM: A Benchmark for Evaluating Machine Theory of Mind in LLM-based Conversational Recommender Systems. arXiv:2511.22275 [cs.AI] https://arxiv.org/abs/2511.22275

work page arXiv 2025
[25]

Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards Emotional Support Dialog Systems. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 3469–3483

2021
[26]

org/10.2307/258792 McCraw, B

Roger C. Mayer, James H. Davis, and F. David Schoorman. 1995. An Integrative Model of Organizational Trust.The Academy of Management Review20, 3 (July 1995), 709. doi:10.2307/258792

work page doi:10.2307/258792 1995
[27]

Mistral. 2024. Mistral NeMo. https://mistral.ai/news/mistral-nemo

2024
[28]

Mistral. 2025. Mistral Small 3.2 24B. https://docs.mistral.ai/models/mistral- small-3-2-25-06

2025
[29]

OpenAI. 2025. GPT-4.1. https://openai.com/index/gpt-4-1/

2025
[30]

OpenAI. 2025. GPT-5. https://openai.com/index/introducing-gpt-5/

2025
[31]

Matthew Riemer, Zahra Ashktorab, Djallel Bouneffouf, Payel Das, Miao Liu, Justin D Weisz, and Murray Campbell. 2024. Position: Theory of Mind Bench- marks are Broken for Large Language Models.arXiv preprint arXiv:2412.19726

work page arXiv 2024
[32]

Kazutoshi Shinoda, Nobukatsu Hojo, Kyosuke Nishida, Saki Mizuno, Keita Suzuki, Ryo Masumura, Hiroaki Sugiyama, and Kuniko Saito. 2025. ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind.arXiv preprint arXiv:2501.08838(2025)

work page arXiv 2025
[33]

Littman, and Joshua B

Michael Shum, Max Kleiman-Weiner, Michael L. Littman, and Joshua B. Tenen- baum. 2019. Theory of Minds: Understanding Behavior in Groups through Inverse Planning.Proceedings of the AAAI Conference on Artificial Intelligence33, 01 (July 2019), 6163–6170. doi:10.1609/aaai.v33i01.33016163

work page doi:10.1609/aaai.v33i01.33016163 2019
[34]

James WA Strachan, Dalila Albergo, Giulia Borghini, Oriana Pansardi, Eugenio Scaliti, Saurabh Gupta, Krati Saxena, Alessandro Rufo, Stefano Panzeri, Guido Manzi, et al. 2024. Testing theory of mind in large language models and humans. Nature Human Behaviour(2024), 1–11

2024
[35]

Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and Others. 2025. Kimi K2: Open Agentic Intelligence. arXiv:2507.20534 [cs.LG] https://arxiv.org/abs/ 2507.20534

work page internal anchor Pith review arXiv 2025
[36]

Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https: //arxiv.org/abs/2505.09388

work page internal anchor Pith review Pith/arXiv arXiv 2025
[37]

Qiaosi Wang, Xuhui Zhou, Maarten Sap, Jodi Forlizzi, and Hong Shen. 2025. Rethinking Theory of Mind Benchmarks for LLMs: Towards A User-Centered Perspective. arXiv:2504.10839 [cs.HC] https://arxiv.org/abs/2504.10839

work page arXiv 2025
[38]

Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Anna Korhonen, David Traum, and Lluís Màrquez (Eds.). Association for Computationa...

work page doi:10.18653/v1/p19-1566 2019
[39]

Alex Wilf, Sihyun Shawn Lee, Paul Pu Liang, and Louis-Philippe Morency. 2023. Think Twice: Perspective-Taking Improves Large Language Models’ Theory-of- Mind Capabilities.arXiv preprint arXiv:2311.10227(2023)

work page arXiv 2023
[40]

H Wimmer. 1983. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception.Cognition13, 1 (Jan. 1983), 103–128. doi:10.1016/0010-0277(83)90004-5

work page doi:10.1016/0010-0277(83)90004-5 1983
[41]

Yufan Wu, Yinghui He, Yilin Jia, Rada Mihalcea, Yulong Chen, and Naihao Deng. 2023. Hi-ToM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models. InFindings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapor...

work page doi:10.18653/v1/2023.findings-emnlp.717 2023
[42]

Zixiu Wu, Simone Balloccu, Vivek Kumar, Rim Helaoui, Ehud Reiter, Diego Reforgiato Recupero, and Daniele Riboni. 2022. Anno-MI: A Dataset of Expert- Annotated Counselling Dialogues. InICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 6177–6181. doi:10. 1109/ICASSP43922.2022.9746035

work page arXiv 2022
[43]

Yang Xiao, Jiashuo Wang, Qiancheng Xu, Changhe Song, Chunpu Xu, Yi Cheng, Wenjie Li, and Pengfei Liu. 2025. Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal Evolution of Human States. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterin...

work page doi:10.18653/v1/2025.acl-long.1171 2025
[44]

Hainiu Xu, Runcong Zhao, Lixing Zhu, Jinhua Du, and Yulan He. 2024. Open- ToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Asso...

work page doi:10.18653/v1/2024.acl-long.466 2024
[45]

Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Zhengyang Qi, Haofei Yu, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, and Maarten Sap. 2024. SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents.ICLR. https://openreview.net/forum?id=mM7VurbA4r

2024
[46]

ground truth

Wentao Zhu, Zhining Zhang, and Yizhou Wang. 2024. Language Models Repre- sent Beliefs of Self and Others. InForty-first International Conference on Machine Learning. A Dataset and Code Availability To ensure reproducibility, our dataset and code is publicly available at the following URL: https://github.com/Stealth-py/DialToM. B Detailed Data Annotation P...

2024