Pith. sign in

REVIEW 3 major objections 6 minor 40 references

The Essence of Contextual Understanding in Theory of Mind: A Study on Question Answering with Story Characters

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that human Theory of Mind relies on long-term knowledge of a person's background, and that current LLMs, including o1 and DeepSeek-R1, perform notably worse than human readers on questions that require this knowledge.

desk verdict A genuinely useful new ToM benchmark, but the LLM-versus-human claim is undercut because the models never receive the global context the task defines. read the letter →

arxiv 2501.01705 v2 pith:2MHKW4IK submitted 2025-01-03 cs.CL cs.AI

Chulun Zhou , Qiujing Wang , Mo Yu , Xiaoqian Yue , Rui Lu , Jiangnan Li , Yifan Zhou , Shunchi Zhang
show 2 more authors
Jie Zhou Wai Lam
This is my paper · ORCID
classification cs.CLcs.AI
keywords TheoryofMindquestionansweringlargelanguagemodelscontextualunderstandingcharacterbenchmarklong-contextreasoningnovelcomprehension
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a core part of human Theory of Mind—the ability to attribute mental states to others—is understanding a person's long-term background and life story, and that existing benchmarks built on short, self-contained stories never test this. To support the point, the authors introduce CharToM-QA, 1,035 questions about characters from 20 classic novels, covering belief, intention, emotion, and desire. In a human study, educated participants answered far better when they had read the novel than when they had not, while even strong LLMs underperformed readers despite the stories being in their training data. If correct, this means long-term contextual understanding is a distinct, mostly missing axis in machine Theory of Mind evaluation.

What carries the argument

The load-bearing object is the CharToM-QA benchmark itself: 1,035 Theory of Mind questions drawn from reader annotations on 20 classic novels, with each question tied to a plot window and to one of four Theory of Mind dimensions (belief, intention, emotion, desire). Because the questions concern characters whose full life stories are spread across an entire book, answering them requires global context rather than only the supplied window. The argument is carried by two comparisons: readers versus non-readers on the same questions, which isolates the contribution of long-term background knowledge, and humans versus LLMs under matched context windows, which tests whether models can substitute scale or reasoning for that knowledge. A secondary mechanism is the generative evaluation protocol, which grades responses by whether they cover extracted bonus points and penalizes factual or logical defects, because it correlates with human judgment better than token- or embedding-based metrics.

What would settle it

A preregistered replication with a larger, balanced sample that finds no reader-versus-non-reader accuracy gap on the same 150 questions would falsify the central claim; a second decisive result would be non-readers matching readers once given a written character-background summary, which would show the effect is information access rather than memory integration.

Watch

Extended reading notes

Core claim

The central discovery the paper reports is that Theory of Mind about a character is inseparable from knowledge of that character's personal history: readers who had read a novel beat non-readers by about 21 percent on the same multiple-choice questions about the characters' beliefs, intentions, emotions, and desires, while GPT-4o, o1, and DeepSeek-R1 all performed below the readers and roughly at or below the non-reader level. The paper also finds that human accuracy rises when people are given a longer story window, but LLM accuracy stays flat as the window grows, and that LLMs drop most on indirect questions whose answers are not literally present in the provided plot. The authors interpret this as evidence that current models rely on local textual clues or memorized story knowledge rather than the integrated, global understanding that human Theory of Mind uses.

Load-bearing premise

The claim that long-term familiarity drives the human advantage rests on a small human study (eight participants, 150 self-reported reading assignments, no significance test on the roughly 21 percent gap), so if that gap does not survive a larger sample the central human-LLM comparison weakens.

Editorial extensions

If this is right

  • Theory of Mind benchmarks built on short, self-contained narratives miss the global-background component that this paper argues is central to human mental-state inference.
  • State-of-the-art LLMs, including reasoning models with chain-of-thought, remain below human readers on these questions and do not improve when given longer story windows.
  • The human advantage appears specifically on indirect questions, where the answer requires knowledge outside the given plot window, and this is exactly where LLM performance drops most.
  • The flat LLM performance across context lengths may reflect reliance on memorized story content rather than genuine contextual integration, which the paper flags as a Clever Hans risk.
  • The paper's grading-style evaluation protocol, based on bonus-point coverage with penalty detection, correlates with human judgment better than ROUGE or BERTScore and offers a reusable metric for generative Theory of Mind QA.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reader-versus-non-reader gap replicates in a larger sample, then current 'Theory of Mind' results on short narratives likely overstate machine social reasoning, because they never engage the long-term person-model component.
  • A natural extension the paper does not run: give non-reader humans or LLMs a written biography of the character before the same questions; if the gap closes, the binding constraint is access to background information, not the ability to integrate it over a long text.
  • Another testable extension is retrieval: measure whether supplying relevant background passages, rather than a longer window, changes LLM accuracy; the paper's stable-window result predicts that retrieval helps only if it provides the causal links models currently miss.
  • The visible chain-of-thought finding that 76 percent of thoughts recall relevant background suggests that even when models locate the right context, they often fail to convert it into correct answers, pointing to integration rather than knowledge access as the bottleneck.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces CHARTOM-QA, a new benchmark of 1,035 theory-of-mind (ToM) multiple-choice and generative questions about characters from classic novels. The benchmark is built with an AI-assisted human annotation pipeline that uses reader notes, GPT-4o for paraphrasing and question generation, and expert validation. The authors report a human study with 8 participants and 150 questions, finding that readers of the novels outperform non-readers by about 21%, and that non-readers perform at roughly the level of GPT-4o. They also evaluate six LLMs (including GPT-4o, GPT-3.5, Llama-3.1, Qwen2, Mistral, InternLM2) and the reasoning models o1 and DeepSeek-R1, finding that even the strongest models lag behind readers. The paper's central claim is that human ToM depends on long-term knowledge of a person's background—knowledge that current LLMs cannot recover from short plot windows despite possible pretraining exposure to the source texts.

Significance. If the central claim holds, the benchmark addresses a genuinely underexplored aspect of ToM evaluation: the need to integrate global character history rather than only local narrative cues. The dataset construction is creative, using authentic reader notes from online reading platforms, and the final benchmark spans diverse classic novels and 150 characters. The paper also proposes a bonus-point/penalty evaluation protocol and reports that it correlates more strongly with human judgments (Pearson 0.92) than Rouge or BERTScore. The inclusion of o1 and DeepSeek-R1 is timely. However, the significance is contingent on the validity of the human-LLM comparison and on the absence of confounds in the benchmark design; the paper's own evidence has several gaps that need to be addressed before the strong claims about LLM limitations can be accepted.

major comments (3)
  1. [§3, §6.2, Appendix K] Equation (1) formally defines the task as P(a|q,W,G), conditioning on the global context G of the whole book, but every LLM experiment in Tables 1–3 and the prompts in Appendix K provide only a plot window W of 0/1k/2k tokens, never G. The abstract's assertion that LLMs 'have seen these stories during pre-training' is not verified for the specific Chinese translations and editions used, and parametric recall of a book is not equivalent to having its global context available during inference. Consequently, the human-LLM comparison in Table 3 is asymmetric: human readers have actual memory of G, while LLMs receive only W. The conclusion that LLMs fail to exploit long-term background is therefore confounded with the absence of G from the model input. The authors should either supply the full book (or a substantial relevant excerpt) to LLMs, add an explicit retrieval step, or probe whether the models can recall the relevant background; otherwise the main LLM claim is not supported.
  2. [§6.3, Table 3] The human study relies on only 8 participants and 150 questions, reports no significance tests, confidence intervals, or inter-rater reliability, and defines reading status by self-report. The ~21% gap between the 'w/. history' and 'w/o. history' groups is load-bearing for RQ1 and for the human-LLM comparison, but with this sample size the gap could be driven by individual differences, by partial cultural familiarity with classic novels, or by chance. The authors should report a mixed-effects model with participant and book random effects, bootstrap confidence intervals, or at least a screening quiz to confirm that the 'read' group actually remembers the books and the 'non-read' group does not. Without such evidence, the claim that book familiarity causes the human advantage is not established.
  3. [§4.2, §5, Appendix E] GPT-4o participates at nearly every stage of the benchmark: it paraphrases user notes into ToM descriptions, generates candidate questions, constructs distractors, extracts bonus points, and scores model responses. This creates a circularity risk: the benchmark may reward the kinds of answers that GPT-4o itself prefers. The reported 0.92 human correlation in Appendix E is computed on a subset, but the number of responses and human judgments, the selection procedure, and confidence intervals are not reported. The authors should report the sample size and design of the correlation study, validate a held-out set of final questions and answers with human experts (not just the underlying notes), and preferably include a non-GPT-4o evaluator (e.g., a smaller open-source LLM or human ratings) to confirm the relative rankings.
minor comments (6)
  1. [Abstract, §1, Table 4] The benchmark name is inconsistent across the manuscript: the abstract and main text use 'CHARTOM-QA' in some places and 'CharToM' in others; please unify the naming.
  2. [Table 3] The abbreviation 'indir.' in Table 3 is not defined until §6.4; please define it directly in the table caption.
  3. [§4.2] There is a typo: 'aformentioned' should be 'aforementioned'.
  4. [Appendix E] The Pearson correlation of 0.92 is reported without the number of model responses or human judgments used; please add these details.
  5. [Table 5] The table uses 'N/A', '–', and '!' symbols without a clear legend in the caption; please explain them explicitly.
  6. [References] The references for Ma et al. 2023a and 2023b are both cited in §2 and appear in the bibliography; please check that all in-text citations (including 'Tomchallenges' and 'TOMBENCH') point to the intended entries.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the paper's claims are empirical benchmark results, not predictions derived from their own inputs.

full rationale

The paper contributes an empirical benchmark and an evaluation, not a derivation that reduces to its inputs. The problem formulation in Eq. 1 defines the intended task P(a|q,W,G), but the central claim that humans outperform LLMs on CharToM-QA is established by direct measurement of multichoice accuracy on human-verified answer choices, not by algebraic or definitional equivalence with Eq. 1. The abstract's statement that LLMs 'have seen these stories during pre-training' is an assumption about parametric memory and is even discussed as a possible Clever Hans effect in Section 6.4; this is a validity concern about whether G is effectively available to the model, but it is not a self-definitional reduction, because the paper never claims to derive LLM failure from the absence of G. GPT-4o is used in question generation, answer paraphrasing, bonus-point extraction, and generative evaluation, which is a legitimate validity risk noted in the Limitations section; however, the human-LLM comparison that anchors the main conclusion uses multichoice accuracy with human-verified answers, and Appendix E reports a 0.92 Pearson correlation between the GPT-4o-based BPC protocol and human judgments on a subset. The self-citations (Yu et al. 2023, Xu et al. 2024b, Yu et al. 2024) are methodological precedents, possible explanations, or prior dataset construction approaches; none is load-bearing in the sense of forcing the conclusion by an unverified author-specific result. No equation, fitted parameter, or cited uniqueness theorem is equivalent to the paper's own output by construction, so the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no free parameters or invented entities. Its central claim rests on domain assumptions about the validity of user notes, the representativeness of four ToM dimensions, and the reliability of GPT-4o as a generator and judge. The human study is the main empirical load-bearing component.

assumptions (4)
  • domain assumption The four ToM dimensions (belief, intention, emotion, desire) are sufficient for the central claim about contextual understanding.
    The paper only tests these four dimensions, which limits the scope of the contextual-ToM claim; other dimensions like knowledge are excluded (Limitations).
  • domain assumption User notes from reading apps are valid proxies for human ToM interpretations of characters.
    The dataset is built on user notes; a manual check found 93% acceptable, but the sample was 100 notes (Section 4.1).
  • domain assumption GPT-4o can generate fluent, faithful questions and evaluate responses reliably.
    GPT-4o is used for question generation, paraphrasing, bonus-point extraction, and scoring; human validation is limited to a subset.
  • domain assumption LLMs have seen the evaluated books during pre-training.
    The paper infers the Clever Hans possibility and stable performance with context length, but actual memorization is not measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Essence of Contextual Understanding in Theory of Mind: A Study on Question Answering with Story Characters." pith.science (2026). https://pith.science/paper/2MHKW4IK

@misc{pith2026250101705,
  author       = {Pith},
  title        = {Pith review of: The Essence of Contextual Understanding in Theory of Mind: A Study on Question Answering with Story Characters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2MHKW4IK}},
  note         = {Machine review of arXiv:2501.01705}
}
read the original abstract

Theory-of-Mind (ToM) is a fundamental psychological capability that allows humans to understand and interpret the mental states of others. Humans infer others' thoughts by integrating causal cues and indirect clues from broad contextual information, often derived from past interactions. In other words, human ToM heavily relies on the understanding about the backgrounds and life stories of others. Unfortunately, this aspect is largely overlooked in existing benchmarks for evaluating machines' ToM capabilities, due to their usage of short narratives without global context, especially personal background of characters. In this paper, we verify the importance of comprehensive contextual understanding about personal backgrounds in ToM and assess the performance of LLMs in such complex scenarios. To achieve this, we introduce CharToM benchmark, comprising 1,035 ToM questions based on characters from classic novels. Our human study reveals a significant disparity in performance: the same group of educated participants performs dramatically better when they have read the novels compared to when they have not. In parallel, our experiments on state-of-the-art LLMs, including the very recent o1 and DeepSeek-R1 models, show that LLMs still perform notably worse than humans, despite that they have seen these stories during pre-training. This highlights the limitations of current LLMs in capturing the nuanced contextual information required for ToM reasoning.

Figures

Figures reproduced from arXiv: 2501.01705 by the authors.

Figure 1
Figure 1. An example from CHARTOM-QA bench￾mark. LLMs make their responses given a question and corresponding story plot in generative and multichoice QA settings. dimensions of belief, intention, emotion, desire, etc) to them (Premack and Woodruff, 1978). For human, ToM holds an important position in daily social interactions, as it enables people to explain and predict the behavior of others with no direct access to their m… view at source ↗
Figure 2
Figure 2. Illustration of notes from users on reading [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Given the corresponding story plot, GPT-4o is used to generate candidate questions about the target [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 11 canonical work pages

  1. [1]

    theory of mind

    Ian Apperly. 2010. Mindreaders: the cognitive basis of" theory of mind". Psychology Press

    2010

  2. [2]

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. 2024. Internlm2 technical report. arXiv preprint arXiv:2403.17297

    arXiv 2024

  3. [3]

    Chunkit Chan, Cheng Jiayang, Yauwai Yim, Zheye Deng, Wei Fan, Haoran Li, Xin Liu, Hongming Zhang, Weiqi Wang, and Yangqiu Song. 2024. https://aclanthology.org/2024.findings-emnlp.244 Negotiationtom: A benchmark for stress-testing machine theory of mind on negotiation surrounding . In Findings of the Association for Computational Linguistics: EMNLP 2024, M...

    work page 2024

  4. [4]

    Zhuang Chen, Jincenzi Wu, Jinfeng Zhou, Bosi Wen, Guanqun Bi, Gongyao Jiang, Yaru Cao, Mengting Hu, Yunghwei Lai, Zexuan Xiong, and Minlie Huang. 2024. https://aclanthology.org/2024.acl-long.847 Tombench: Benchmarking theory of mind in large language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volum...

    work page 2024

  5. [5]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  6. [6]

    Murray J Dyck, Kara Ferguson, and Ian M Shochet. 2001. Do autism spectrum disorders differ from each other and from non-spectrum disorders on emotion recognition tests? European child & adolescent psychiatry, 10:105--116

    work page 2001

  7. [7]

    Kanishk Gandhi, Jan - Philipp Fr \" a nken, Tobias Gerstenberg, and Noah D. Goodman. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/2b9efb085d3829a2aadffab63ba206de-Abstract-Datasets\_and\_Benchmarks.html Understanding social reasoning in language models with language models . In Advances in Neural Information Processing Systems 36: Annual Confe...

    work page 2023

  8. [8]

    Griffiths

    Erin Grant, Aida Nematzadeh, and Thomas L. Griffiths. 2017. https://mindmodeling.org/cogsci2017/papers/0088/index.html How can memory-augmented neural networks pass a false-belief task? In Proceedings of the 39th Annual Meeting of the Cognitive Science Society, CogSci 2017, London, UK, 16-29 July 2017. cognitivesciencesociety.org

    work page 2017

Show all 40 references
  1. [9]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  2. [10]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  3. [11]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L \' e lio Renard Lavaud, Marie - Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, ...

    2023 doi arXiv

  4. [12]

    Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi, and Maarten Sap. 2023. https://doi.org/10.18653/V1/2023.EMNLP-MAIN.890 Fantom: A benchmark for stress-testing machine theory of mind in interactions . In Proceedings of the 2023 Conference on Empiri...

    2023 doi

  5. [13]

    Tom \'a s Ko c isk \`y , Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, G \'a bor Melis, and Edward Grefenstette. 2018. https://aclanthology.org/Q18-1023.pdf The narrativeqa reading comprehension challenge . Transactions of the Association for Computational L...

    2018

  6. [14]

    Michal Kosinski. 2023. https://doi.org/10.48550/ARXIV.2302.02083 Theory of mind may have spontaneously emerged in large language models . CoRR, abs/2302.02083

    2023 doi arXiv

  7. [15]

    a ldchen, Alexander Binder, Gr \' e goire Montavon, Wojciech Samek, and Klaus - Robert M \

    Sebastian Lapuschkin, Stephan W \" a ldchen, Alexander Binder, Gr \' e goire Montavon, Wojciech Samek, and Klaus - Robert M \" u ller. 2019. https://arxiv.org/abs/1902.10178 Unmasking clever hans predictors and assessing what machines really learn . CoRR, abs/1902.10178

    2019 arXiv

  8. [16]

    Matthew Le, Y - Lan Boureau, and Maximilian Nickel. 2019. https://doi.org/10.18653/V1/D19-1598 Revisiting the evaluation of theory of mind through question answering . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Interna...

    2019 doi

  9. [17]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  10. [18]

    Xiaomeng Ma, Lingyu Gao, and Qihui Xu. 2023 a . https://doi.org/10.18653/V1/2023.CONLL-1.2 Tomchallenges: A principle-guided dataset and diverse evaluation tasks for exploring theory of mind . In Proceedings of the 27th Conference on Computational Natural Language Learning, Co...

    2023 doi

  11. [19]

    Ziqiao Ma, Jacob Sansom, Run Peng, and Joyce Chai. 2023 b . https://doi.org/10.18653/v1/2023.findings-emnlp.72 Towards a holistic landscape of situated theory of mind in large language models . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 101...

    2023 doi

  12. [20]

    Griffiths

    Aida Nematzadeh, Kaylee Burns, Erin Grant, Alison Gopnik, and Thomas L. Griffiths. 2018. https://doi.org/10.18653/V1/D18-1261 Evaluating theory of mind in question answering . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, ...

    2018 doi

  13. [21]

    Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, and Samuel Bowman. 2022. https://doi.org/10.18653/v1/2022.naacl-main.391 Q u ALITY : Question answering with long input texts, ye...

    2022 doi

  14. [22]

    David Premack and Guy Woodruff. 1978. Does the chimpanzee have a theory of mind? Behavioral and brain sciences, 1(4):515--526

    1978

  15. [23]

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. https://doi.org/10.18653/V1/D19-1454 Social iqa: Commonsense reasoning about social interactions . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9t...

    2019 doi

  16. [24]

    Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2024. https://aclanthology.org/2024.eacl-long.138 Clever hans or neural theory of mind? stress testing social reasoning in large language models . In Proceed...

    2024

  17. [25]

    Natalie Shapira, Guy Zwirn, and Yoav Goldberg. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-ACL.663 How well do large language models perform on faux pas tests? In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023 , pages ...

    2023 doi

  18. [26]

    Katherine Thai, Yapei Chang, Kalpesh Krishna, and Mohit Iyyer. 2022. https://doi.org/10.18653/V1/2022.ACL-LONG.517 Relic: Retrieving evidence for literary claims . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers...

    2022 doi

  19. [27]

    Diab, Bonnie J

    Jennifer Tracey, Owen Rambow, Claire Cardie, Adam Dalton, Hoa Trang Dang, Mona T. Diab, Bonnie J. Dorr, Louise Guthrie, Magdalena Markowska, Smaranda Muresan, Vinodkumar Prabhakaran, Samira Shaikh, and Tomek Strzalkowski. 2022. https://aclanthology.org/2022.lrec-1.262 Best: Th...

    2022

  20. [28]

    Mengting Wan, Rishabh Misra, Ndapa Nakashole, and Julian J. McAuley. 2019. https://doi.org/10.18653/V1/P19-1248 Fine-grained spoiler detection from large-scale review corpora . In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Fl...

    2019 doi

  21. [29]

    Heinz Wimmer and Josef Perner. 1983. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children's understanding of deception. Cognition, 13(1):103--128

    1983

  22. [30]

    Jincenzi Wu, Zhuang Chen, Jiawen Deng, Sahand Sabour, Helen Meng, and Minlie Huang. 2024. https://doi.org/10.18653/V1/2024.ACL-LONG.848 COKE: A cognitive knowledge graph for machine theory of mind . In Proceedings of the 62nd Annual Meeting of the Association for Computational...

    2024 doi

  23. [31]

    Yufan Wu, Yinghui He, Yilin Jia, Rada Mihalcea, Yulong Chen, and Naihao Deng. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.717 Hi-tom: A benchmark for evaluating higher-order theory of mind reasoning in large language models . In Findings of the Association for Comput...

    2023 doi

  24. [32]

    Hainiu Xu, Runcong Zhao, Lixing Zhu, Jinhua Du, and Yulan He. 2024 a . https://doi.org/10.18653/V1/2024.ACL-LONG.466 Opentom: A comprehensive benchmark for evaluating theory-of-mind reasoning capabilities of large language models . In Proceedings of the 62nd Annual Meeting of ...

    2024 doi

  25. [33]

    Liyan Xu, Jiangnan Li, Mo Yu, and Jie Zhou. 2024 b . https://doi.org/10.18653/v1/2024.acl-long.317 Fine-grained modeling of narrative context: A coherence perspective via retrospective questions . In Proceedings of the 62nd Annual Meeting of the Association for Computational L...

    2024 doi

  26. [34]

    Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Li, Nora Bradford, Branda Sun, Tran Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, and Mark Warschauer. 2022. https://doi.org/10.18653/v1/2022.acl-long.34 F...

    2022 doi

  27. [35]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  28. [36]

    Mo Yu, Jiangnan Li, Shunyu Yao, Wenjie Pang, Xiaochen Zhou, Zhou Xiao, Fandong Meng, and Jie Zhou. 2023. Personality understanding of fictional characters during book reading. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1:...

    2023

  29. [37]

    Mo Yu, Qiujing Wang, Shunchi Zhang, Yisi Sang, Kangsheng Pu, Zekai Wei, Han Wang, Liyan Xu, Jing Li, Yue Yu, and Jie Zhou. 2024. https://openreview.net/forum?id=ZZ7UKgK4c1 Few-shot character understanding in movies as an assessment to meta-learning of theory-of-mind . In Forty...

    2024

  30. [38]

    McKee, Ari Holtzman, Jay Pujara, Xiang Ren, Swaroop Mishra, Aida Nematzadeh, Shyam Upadhyay, and Manaal Faruqui

    Pei Zhou, Aman Madaan, Srividya Pranavi Potharaju, Aditya Gupta, Kevin R. McKee, Ari Holtzman, Jay Pujara, Xiang Ren, Swaroop Mishra, Aida Nematzadeh, Shyam Upadhyay, and Manaal Faruqui. 2023. https://doi.org/10.48550/ARXIV.2310.03051 How far are large language models from age...

    2023 doi arXiv

  31. [39]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  32. [40]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.