Computation and Language · arXiv:2610.18610 v1 · submitted 13 Oct 2026 · 56 references
Jev-ToM: Evaluating Theory of Mind in Typed Decision Models
The paper says
“Dialogue and collaborative assistants repeatedly judge what a partner knows, believes, or intends. Typed decision models offer a potential alternative to generative large language models for bounded Theory of Mind (ToM) selection.”
more of the abstract
We present Jev-ToM, comparing five typed decision models and seven generative baselines on 11,117 English text questions from nine ToM benchmarks across six capability groups. TypeSafe Jev reaches 73.2% source-balanced accuracy including native controls and ranks first or second across all six capabilities, leading beliefs, goals/desires/intentions, and emotions. Other typed models show distinct strengths across tasks. Separate resource comparisons suggest that Jev could reduce the cost and waiting time of repeated ToM selection. Confidence-based selection remains benchmark-dependent, and supplied reasons can repair errors or overturn correct answers depending on their attachment. Candidate expansion reveals distinct selection and output-validity problems. Related-question cues account for most of Jev's downstream gains, with task-dependent effects from predicted states. The findings motivate testing Jev as a ToM selection component in assistants, with task-specific confidence calibration, candidate design, and output validation as integration priorities.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 14 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A transparent, careful component evaluation that establishes Jev as a competitive bounded ToM selector; the headline 'top-two in all six capabilities' is the softest claim, since two of those ranks rest on margins under a point and a mapping layer the authors admit was not independently validated. This line is wrong
2 major objections · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 56 references say.
The paper says, at Section 3.2 / Appendix B.1; Tables 5, 8-9; Figure 2
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.86):
“This was not independent hu- man validation.”
Pith says: Objection 1 of 2 · major
The headline claim that Jev 'ranks first or second across all six capabilities' (Abstract; Section 4.1) rests on the assignment of questions to capability groups, and for all 730 ToMBench items this assignment was produced by an agent-assisted post-evaluation review that the paper itself describes as 'not independent human validation' (Section 3.2; Appendix B.1).
more of objection 1
The per-item mapping decisions are deferred to the planned reproduction release (Appendix I), so the mapping cannot be audited at item level from the manuscript or the prepared review archive. The ranking claim is sensitive to this mapping at nearly single-item scale: in Table 8 Jev is second in mental-state application by 0.3 points (70.4 vs 70.7), and since the application capability is averaged over three sources, a single ToMBench application item contributes roughly 100/(3 x 152) = 0.22 points to that average; the social pragmatics gap (89.1 vs 89.9) is on the order of three items across the two contributing sources, and the 152 ToMBench application assignments overlap the state categories (Appendix B.1). A small number of relabelings can therefore change the top-two sets in at least two capability rows. The acknowledgment in Section 3.2 identifies the assumption, but it does not resolve the risk. I request that the revision (i) include the full per-item mapping in the review materials rather than in a planned release, (ii) provide an independent human-validation sample of the ToMBench mapping labels, and (iii) report rank stability under plausible mapping perturbations, for example by re-assigning a small fraction of the ToMBench labels and recomputing the six capability rankings.
The paper says, at Section 3.3 / Appendix B.5; Tables 8-9; Figure 2
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
Even holding the capability mapping fixed, the six rank claims are reported as point estimates with no uncertainty, and the family-bootstrap protocol of Appendix B.5 is not applied to the capability rows of Tables 8 and 9. Three of the six ranks are decided by margins of 0.3, 0.8, and 2.5 points, which are small relative to the family-level variation the paper itself quantifies for the overall comparisons (e.g., the control-free contrast is -1.79 [-3.48, -0.07] in Table 10).
more of objection 2
Since the rank claim is the paper's headline, I ask for bootstrap intervals on the per-capability accuracy differences, at least for the second-place rows, or an explicit statement that the ranks are point estimates without a sampling-based stability assessment. This request is distinct from the mapping-auditability issue in Major Comment 1: a bootstrap analysis would address sampling variation given the current mapping, while a perturbation analysis would address the mapping itself.
5 minor objections
The paper says, at Abstract / Section 3.3
Pith says: minor objection 1 of 5
The abstract juxtaposes the 73.2% source-balanced aggregate (Eq. 2, which includes native controls and the 505 unmapped CogToM items) with the six-capability ranks (Eq. 3, which excludes controls and unmapped items) without noting that the two numbers are computed on different subsets; one clarifying clause would prevent readers from attributing the rank claim to the 11,117-question aggregate.
The paper says, at Abstract / Section 4.1
Pith says: minor objection 2 of 5
Because removing the 920 native controls reverses Jev's overall lead over GPT-3.5 (69.1% vs 70.9%, Table 10, paired difference -1.79 [-3.48, -0.07]), I recommend surfacing the control-free figure in the abstract or in the opening of Section 4.1 rather than leaving it to Table 10, so that the headline's dependence on control inclusion is visible.
show 3 more
The paper says, at Section 4.3 / Figure 5
Pith says: minor objection 3 of 5
Section 4.3 opens with the large repair magnitudes from the error-enriched reused materials (Figure 4), while the fresh-material result for Jev is much smaller (+3.7 points aligned, net -0.40 [-3.03, 2.17] crossed; Figure 5 and Table 13); a sentence at the start of the section flagging the reused/new split as the primary interpretative key would reduce the risk of over-reading Figure 4.
The paper says, at Tables 8-9 / Figure 2
Pith says: minor objection 4 of 5
Tables 8 and 9 pack twelve model columns with no special marking of the cross-source rows that carry the headline rank claim; I recommend highlighting the top-two cells in those rows as in Table 2 and making a machine-readable version of the tables available with the revision.
The paper says, at Section 3.2 / Appendix B.2
Pith says: minor objection 5 of 5
The subset provenance is described only as 'fixed development or previously used subsets'; for reproducibility, please state for each of the nine benchmarks which official split or previously used subset is adopted, citing the source of the subset construction where it is inherited from prior work such as SimToM.
What would settle it
Have two non-author annotators, blind to all model predictions and to Jev's results, independently redo the six-way capability mapping for all 730 ToMBench targets plus all items from the other sources that Table 8 assigns to social pragmatics or mental-state application, using the paper's stated group definitions. Recompute the six capability rows of Table 8 with the frozen primary predictions and the independently derived mapping. If Jev remains first or second in all six rows and the top-two sets in social pragmatics and mental-state application are unchanged, the concern is settled. If any top-two set changes, the headline capability claim is not established under an independent mapping.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 14 Oct 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Worth your time. This is the first dedicated cross-benchmark ToM evaluation of Jev, and it is more careful than the typical eval paper in this space. The protocol is strict—exact match, invalid outputs count as errors—key comparisons carry family-bootstrap intervals, the authors report a sensitivity analysis in which removing native controls reverses Jev's lead over GPT-3.5 (69.1 vs 70.9), and the fixed-No replay in the feedback study shows related-question cues, not instance-specific predicted states, drive most of Jev's downstream gain. That last control is the kind of thing people usually skip. The reason-attachment and repair/damage decompositions are also solid evaluation practice.
The central claim holds up: Jev is a competitive bounded ToM selector at lower reference cost and observed latency. The 'ranks first or second across all six capabilities' headline is the one claim to qualify. Two of those ranks sit on thin margins—0.8 points in social pragmatics and 0.3 in mental-state application over DeepSeek—and the capability mapping for ToMBench's 730 items was agent-assisted, which the paper explicitly says was not independent human validation. The mapping rules and per-capability counts are in the appendix, but the item-level assignments are not auditable until the reproduction package ships, and a small number of re-labeled items could flip those rows. I'd note the concern is bounded: 10,387 of the 11,117 questions map through published native labels; only ToMBench depends on the agent-assisted layer.
Other soft spots, in order: the reproduction package is planned, not released; per-benchmark uncertainty estimates cover only key comparisons; and the latency comparison mixes serial and concurrent serving with unrecorded hardware, which Appendix G acknowledges. The cost comparison, using frozen reference tariffs, is cleaner. None of this breaks the central result, but it keeps the paper at 'conditional' rather than settled.
Who it's for: people building assistants that need cheap repeated mental-state selection, and researchers evaluating typed decision models. The authors correctly scope it as component-level, not full-assistant evidence.
Recommendation: send it to review. The referee should push for the release and an independent check of the ToMBench mapping, but this is a serious, honest empirical study, not a desk reject.
The strongest objection
The central claim has two parts: the 73.2% source-balanced accuracy and the capability-level ranks. The ranking part is the more load-bearing because it is the paper's headline. For that part to hold, each item must be assigned to the right capability group. The benchmark reference answers come from published, citable sources, so I do not treat those as suspect. The less secure link is the authors' own mapping layer: Section 3.2 and Appendix B.1 state that all 730 ToMBench targets underwent an agent-assisted post-evaluation review, and the paper explicitly says this was not independent human validation (b71). The per-item mapping decisions are deferred to the planned reproduction materials, so the mapping cannot currently be checked from the paper alone. The ranking claim is sensitive to this mapping in a concrete way. In Table 8, Jev is second in social pragmatics by 0.8 points (89.1 vs DeepSeek's 89.9) and second in mental-state application by 0.3 points (70.4 vs DeepSeek's 70.7); knowledge access is second by 2.5 points. The ToMBench application assignments (152 items) overlap the state categories, so a small number of re-labeled items changes both rows. A modest fraction of items moving between groups can therefore change the top-two sets. The authors acknowledge the non-independence in Section 3.2, but the acknowledgment identifies the assumption the headline rests on; it does not resolve it.
The reader's weakest assumption and my own concern converge on the capability mapping, so I agree with the reader. I read the paper as a transparent empirical evaluation rather than a claim of architectural superiority: the authors repeatedly qualify that the confidence analyses are exploratory, that the cutoff is not a validated policy, that serving conditions differ, that removing native controls reverses the aggregate ordering, and that the reason materials are experimental candidate text. Those acknowledged limitations are not the load-bearing issue. The claim that would have to be true for the central result is that the six capability groups measure what they are labelled as measuring. Published references are a legitimate basis for the answer keys, and the authors are not required to re-prove those benchmarks. But the mapping from native labels to ATOMS groups is the authors' own construction, and for ToMBench it was reviewed by an agent rather than independently validated by humans. The paper says so explicitly (b71). Since the headline ranking claim is made at the level of these groups, the mapping is load-bearing. The sensitivity is concrete, not hypothetical. The two rows in which Jev is second by less than one percentage point are social pragmatics and mental-state application. Both are small aggregates (383 and 946 items) whose composition depends on the disputed mapping step; social pragmatics is drawn from only two sources, one of which is ToMBench. If a modest fraction of items were re-assigned, the top-two sets could change. A family-bootstrap interval would quantify sampling uncertainty under the current mapping, but it would not test whether the mapping itself is right; only an independent mapping exercise can do that. I therefore recommend keeping the reader's CONDITIONAL verdict. The concern is a validation condition, not a demonstrated error: the numbers may be exactly right, and the proposed re-annotation is a feasible check. If the independent mapping reproduces Jev's top-two positions, the central claim is considerably stronger. If it does not, the aggregate capability claim is not established. No verdict change is needed beyond the reader's existing conditional recommendation.
The referee's report
If the headline results hold, this is a timely and useful contribution: to my knowledge it is the first multi-benchmark evaluation of typed decision models for ToM selection, and it provides concrete evidence that a compact typed model can be cost- and latency-competitive with generative baselines in that role. The methodological strengths are substantial: the strict scoring contract (Section 3.3) separates selection errors from output-validity failures; the family-level bootstrap (Appendix B.5) is appropriate for the clustered benchmark structure; the fixed-No replay control (Appendix F.2) disciplines the mental-state feedback claim; and the separation of reused versus new reason materials (Figure 5, Table 13) prevents over-reading of the repair results. The paper is also unusually transparent about limitations it has identified, including the control-free reversal (Section 4.1), the acknowledged non-validation of the confidence cutoff (Section 4.2), and the exploratory status of the reason analyses. The claims are stated sharply enough to be falsifiable once the planned data release is in place.
Pith's reading of the paper (a machine's summary, not the authors')
This paper argues that a typed decision model — a system that returns a choice and per-candidate probabilities from a fixed menu instead of generating open-ended text — can carry the bounded Theory of Mind (ToM) judgments that dialogue and collaborative assistants currently make with large generative language models. It compares five typed decision models and seven generative baselines on 11,117 English questions from nine ToM benchmarks, grouped into six capability areas, and reports that TypeSafe Jev reaches 73.2% source-balanced accuracy and ranks first or second in every capability, leading beliefs, goals/desires/intentions, and emotions. The same evidence maps where the model is not yet dependable: confidence-based selection retains 34.9% error on SimpleToM and 29.0% on OpenToM at a 0.90 cutoff, supplied reasons can repair errors or overturn correct answers depending on their attachment, and removing the 920 native control questions reverses Jev's lead over GPT-3.5 (69.1% against 70.9%). If the accuracy picture holds, Jev fits the bounded ToM selection role: its deployments show 73.0% accuracy at 282 ms mean response time, with reference costs 32.7% below GPT-4.1-nano and 56.3% below GPT-4o-mini while scoring higher than both.
What carries the argument. The load-bearing object is the typed decision contract: Jev encodes the story context as state and the question as instructions, and must return a choice plus probabilities that exactly cover the offered candidates, lie in $[0,1]$, sum to one within $10^{-4}$, and give the selected choice a probability within $10^{-6}$ of the maximum. This contract is what makes bounded judgments cheap, fast, and machine-readable, and it is also what lets the paper separate selection errors from output-validity failures, as with the 448 Jev probability-sum failures among 9,522 crossed responses. The second mechanism is the harmonization framework Jev-ToM: source adapters standardize nine benchmarks into records with frozen candidate order and shared family identifiers, native labels are mapped onto six capability groups adapted from the ATOMS taxonomy, and scores are computed with equal benchmark weighting, so a ranking like "first or second in all six capabilities" is an aggregate over task constructions rather than a pooled question count.
Core claim. The paper's central claim is that Jev, a typed decision model, is a competitive bounded Theory of Mind selection component. Given context, a question, and candidate answers, Jev returns a choice with candidate probabilities; on 11,117 questions from nine benchmarks it reaches 73.2% source-balanced exact-match accuracy including native controls and ranks first or second in all six capability groups — knowledge access, beliefs, goals/desires/intentions, emotions, social pragmatics, and mental-state application — with its largest lead in emotions, 84.8% against the strongest generative baseline's 79.7%. The claim carries its own boundary conditions: removing the 920 native control questions reverses Jev's lead over GPT-3.5 (69.1% vs. 70.9%, driven heavily by ToMi belief questions), confidence-based selection at a 0.90 probability cutoff still retains 34.9% error on SimpleToM and 29.0% on OpenToM, and misattached supplied reasons can overturn otherwise correct selections, dropping fresh accuracy on earlier-correct cases from 99.0% to 25.3%. The paper concludes that Jev combines this accuracy with favorable resource use on its deployments — 73.0% accuracy at 282 ms mean response time, and reference costs 32.7% below GPT-4.1-nano and 56.3% below GPT-4o-mini — while naming confidence calibration, candidate design, and output validation as the integration priorities before use within complete assistants.
The premise it rests on. Every accuracy and ranking figure treats each benchmark's stored reference answer and native ability label as the correct ground truth for the mental-state demand being tested, and for ToMBench the capability mapping rests on an agent-assisted review that the paper explicitly says was not independent human validation; if those references or labels are wrong in any systematic way, the claim that Jev ranks first or second in all six capabilities is not established.
If the paper is right (Pith's inferences, not the authors')
Jev can serve as a bounded selection component in assistants: candidate generation and response composition stay with a generative system, while Jev repeatedly picks among finite mental-state interpretations of a partner's knowledge, beliefs, intentions, or emotions.
Deploying Jev for repeated ToM selection would cut both cost and waiting time: at reference prices it costs 32.7% less than GPT-4.1-nano and 56.3% less than GPT-4o-mini with higher accuracy, and its 282 ms mean response time is the shortest in the cohort.
Confidence scores must be calibrated per benchmark before use as an acceptance policy: at the 0.90 cutoff, seven benchmarks fall below 10% retained error while SimpleToM and OpenToM retain 34.9% and 29.0% error at similar coverage.
Candidates should be validated for attachment and size: swapped reason attachments overturn correct selections, and expanding an eight-pair answer-and-reason candidate set to sixteen lowers Jev's strict joint accuracy from 81.0% to 76.4% even though the correct pair remains available.
Question framing, not instance-specific mental-state predictions, drives most of Jev's downstream gain on action prediction: related-question cues account for 10.75 of the 14.5-point improvement, and a post-hoc fixed-No replay on SimpleToM reproduces every predicted-feedback selection at 88.5%.
The headline rank depends partly on how native control questions are scored: with those 920 questions removed, GPT-3.5 edges Jev (70.9% vs. 69.1%), so a deployment-focused replication would probably adopt the control-free aggregate as its primary comparison, since assistants will rarely face benchmark-style controls.
The fixed-No replay result hints at a cheap design rule for builders: on SimpleToM-style awareness-to-behavior tasks, a constant default belief (the partner lacks the information) may capture most of the value of predicted-state feedback, so upstream mental-state prediction should be tested against constant baselines before being added to a pipeline.
Because confidence separates errors far better on some benchmarks than others, a practical routing policy would learn a per-task-family threshold, or escalate low-confidence items to a generative model, rather than apply a single cutoff globally; the paper's data set up this test but do not run it.
The same evaluation shape — frozen candidates, strict output-validity scoring, confidence retention curves, and reason-attachment interventions — transfers to other repeated intermediate judgments in assistants, such as retrieval-record relevance or rubric scoring, where the open question is whether typed selection beats generative generation on the accuracy-cost-latency frontier.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Dialogue and collaborative assistants repeatedly judge what a partner knows, believes, or intends. Typed decision models offer a potential alternative to generative large language models for bounded Theory of Mind (ToM) selection. We present Jev-ToM, comparing five typed decision models and seven generative baselines on 11,117 English text questions from nine ToM benchmarks across six capability groups. TypeSafe Jev reaches 73.2% source-balanced accuracy including native controls and ranks first or second across all six capabilities, leading beliefs, goals/desires/intentions, and emotions. Other typed models show distinct strengths across tasks. Separate resource comparisons suggest that Jev could reduce the cost and waiting time of repeated ToM selection. Confidence-based selection remains benchmark-dependent, and supplied reasons can repair errors or overturn correct answers depending on their attachment. Candidate expansion reveals distinct selection and output-validity problems. Related-question cues account for most of Jev's downstream gains, with task-dependent effects from predicted states. The findings motivate testing Jev as a ToM selection component in assistants, with task-specific confidence calibration, candidate design, and output validation as integration priorities.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-14
The authors' main claim, as the classifier picked it out 0.73
“TypeSafe the classifier reaches 73.2% source-balanced accuracy including native controls and ranks first or second across all six capabilities, leading beliefs, goals/desires/intentions, and emotions.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.90 · human participants 0.12 · controlled comparison 0.95 · randomized assignment 0.03 · states quantitative result 0.99 · compares with prior results 0.96 · mentions code or data availability 0.04
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Dialogue and collaborative assistants repeatedly judge what a partner knows, believes, or intends. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| Typed decision models offer a potential alternative to generative large language models for bounded Theory of Mind (ToM) selection. | background 0.96 | 0.06 | 0.04 | hedged possibility 1.00 |
| We present Jev-ToM, comparing five typed decision models and seven generative baselines on 11,117 English text questions from nine ToM benchmarks across six capability groups. | contribution 0.80 | 0.96 | 0.06 | asserted 1.00 |
| TypeSafe Jev reaches 73.2% source-balanced accuracy including native controls and ranks first or second across all six capabilities, leading beliefs, goals/desires/intentions, and emotions. | result 1.00 | 0.95 | 0.59 | asserted 1.00 |
| Other typed models show distinct strengths across tasks. | result 0.99 | 0.80 | 0.04 | asserted 1.00 |
| Separate resource comparisons suggest that Jev could reduce the cost and waiting time of repeated ToM selection. | result 0.86 | 0.84 | 0.06 | hedged possibility 0.99 |
| Confidence-based selection remains benchmark-dependent, and supplied reasons can repair errors or overturn correct answers depending on their attachment. | result 0.94 | 0.84 | 0.07 | asserted 1.00 |
| Candidate expansion reveals distinct selection and output-validity problems. | result 0.96 | 0.85 | 0.04 | asserted 0.99 |
| Related-question cues account for most of Jev's downstream gains, with task-dependent effects from predicted states. | result 0.86 | 0.91 | 0.10 | asserted 1.00 |
| The findings motivate testing Jev as a ToM selection component in assistants, with task-specific confidence calibration, candidate design, and output validation as integration priorities. | future work 1.00 | 0.28 | 0.50 | hedged possibility 0.72 |
What kind of paper the classifier reports this is: empirical study 0.64 · evidence from observation or experiment 0.86
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-14: 1 request, 7,395 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (10)
Figures are the authors', reproduced from the paper.
References (56)
- [1] Gandhi, Kanishk and Fränken, Jan-Philipp and Gerstenberg, Tobias and Goodman, Noah D. , year =. arXiv preprint arXiv:2306.15448 , url = on PithPith's reviewarXiv
- [2] Tong, Haibo and Yue, Zeyang and Zhao, Feifei and Lin, Erliang and Jia, Lu and Chen, Ruolin and Sun, Yinqian and Zhang, Qian and Zeng, Yi. C og T o M : A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models. Proceedings of the 64th Annual... on Pithdoi:10.18653/v1/2026.acl-long.1448
- [3] Ibrahim, Hazem and Zaki, Yasir , year =. arXiv preprint arXiv:2609.24574 , url = on PithPith's reviewarXiv
- [4] Kim, Hyunwoo and Sclar, Melanie and Zhou, Xuhui and Le Bras, Ronan and Kim, Gunhee and Choi, Yejin and Sap, Maarten. FANT o M : A Benchmark for Stress-testing Machine Theory of Mind in Interactions. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pr... on Pithdoi:10.18653/v1/2023.emnlp-main.890
- [5] Wu, Yufan and He, Yinghui and Jia, Yilin and Mihalcea, Rada and Chen, Yulong and Deng, Naihao. Hi- T o M : A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023. do... on Pithdoi:10.18653/v1/2023.findings-emnlp.717
- [6] Xu, Zixiang and Song, Zirui and Zhang, Chiyu and Chen, Xiuying and Liu, Xi and Hu, Xiyang and Zhao, Yue , year =. arXiv preprint arXiv:2609.30243 , url = on PithPith's reviewarXiv
- [7] Michal Kosinski , journal=. 2024 , doi= on Pith
- [8] Wang, Kevin and Thöni, Anna and Kempinski, Benjamin and Cheng, Bobby and Yao, Jianzhu and Finch, Benjamin and Guertler, Leon and Nadkarni, Viraj and Jiang, Yihan and Korshuk, Aliaksei and Buyantuev, Alexander and Makarov, Ilya and Wu, Siyuan and Cheng, Yu-Chi and Ju, Yan-Ru an... on PithPith's reviewarXiv
- [9] Xu, Hainiu and Zhao, Runcong and Zhu, Lixing and Du, Jinhua and He, Yulan. O pen T o M : A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguis... on Pithdoi:10.18653/v1/2024.acl-long.466
- [10] Gu, Yuling and Tafjord, Oyvind and Kim, Hyunwoo and Moore, Jared and Le Bras, Ronan and Clark, Peter and Choi, Yejin , year =. International Conference on Learning Representations , url = on Pith
- [11] Wilf, Alex and Lee, Sihyun and Liang, Paul Pu and Morency, Louis-Philippe. Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper... on Pithdoi:10.18653/v1/2024.acl-long.451
- [12] Does the chimpanzee have a theory of mind? , author =. Behavioral and Brain Sciences , year =. doi:10.1017/S0140525X00076512 , url = on Pithdoi:10.1017/s0140525x00076512
- [13] Strachan, James W. A. and Albergo, Dalila and Borghini, Giulia and Pansardi, Oriana and Scaliti, Eugenio and Gupta, Saurabh and Saxena, Krati and Rufo, Alessandro and Panzeri, Stefano and Manzi, Guido and Graziano, Michael S. A. and Becchio, Cristina , year =. Nature Human Beh... on Pith
- [14] Shinoda, Kazutoshi and Hojo, Nobukatsu and Nishida, Kyosuke and Mizuno, Saki and Suzuki, Keita and Masumura, Ryo and Sugiyama, Hiroaki and Saito, Kuniko , year =. arXiv preprint arXiv:2501.08838 , url = on PithPith's reviewarXiv
- [15] Beaudoin, Cindy and Leblanc,. Frontiers in Psychology , volume =. 2020 , doi = on Pith
- [16] Chen, Zhuang and Wu, Jincenzi and Zhou, Jinfeng and Wen, Bosi and Bi, Guanqun and Jiang, Gongyao and Cao, Yaru and Hu, Mengting and Lai, Yunghwei and Xiong, Zexuan and Huang, Minlie. T o MB ench: Benchmarking Theory of Mind in Large Language Models. Proceedings of the 62nd Ann... on Pithdoi:10.18653/v1/2024.acl-long.847
- [17] Le, Matthew and Boureau, Y-Lan and Nickel, Maximilian. Revisiting the Evaluation of Theory of Mind through Question Answering. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language ... on Pithdoi:10.18653/v1/d19-1598
- [18] Sun, Yu and Xu, Junhao and Shi, Jiajia and Yang, Zijin , year =. arXiv preprint arXiv:2609.26758 , url = on PithPith's reviewarXiv
- [19] Ullman, Tomer , year =. arXiv preprint arXiv:2302.08399 , url = on PithPith's reviewarXiv
- [20] Chan, Chunkit and Jiayang, Cheng and Yim, Yauwai and Deng, Zheye and Fan, Wei and Li, Haoran and Liu, Xin and Zhang, Hongming and Wang, Weiqi and Song, Yangqiu. N egotiation T o M : A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding. Findings of t... on Pithdoi:10.18653/v1/2024.findings-emnlp.244
show 36 more
- [21] Guangren Wang , year =. arXiv preprint arXiv:2609.36059 , url = on PithPith's reviewarXiv
- [22] Yida Lin , year =. arXiv preprint arXiv:2609.35865 , url = on PithPith's reviewarXiv
- [23] Bushra Asseri and Abdulaziz Asseri , year =. arXiv preprint arXiv:2609.36399 , url = on PithPith's reviewarXiv
- [24] Deu. 2026 , journal = on Pith
- [25] Duan, Feiyu and Lin, Jiayu and Wang, Jia and Xiang, Jun and Wu, Jialiang and Zhang, Xinnong and Yan, Hanqi and Wang, Siyuan and Wei, Zhongyu , year =. arXiv preprint arXiv:2610.03935 , url = on PithPith's reviewarXiv
- [26] Li, Yubo and Miao, Yidi and Krishnan, Ramayya and Padman, Rema , year =. arXiv preprint arXiv:2609.26550 , note = on PithPith's reviewarXiv
- [27] Wang, Lei and Ma, Chen and Feng, Xueyang and Zhang, Zeyu and Yang, Hao and Zhang, Jingsen and Chen, Zhiyuan and Tang, Jiakai and Chen, Xu and Lin, Yankai and Zhao, Wayne Xin and Wei, Zhewei and Wen, Ji-Rong , year =. arXiv preprint arXiv:2308.11432 , url = on PithPith's reviewarXiv
- [28] Jike Zhong and Ming Li and Yuxiang Lai , year =. arXiv preprint arXiv:2609.39496 , url = on PithPith's reviewarXiv
- [29] Han Chen and Yingrui Li , year =. arXiv preprint arXiv:2609.37470 , url = on PithPith's reviewarXiv
- [30] Li, Huao and Chong, Yu Quan and Stepputtis, Simon and Campbell, Joseph and Hughes, Dana and Lewis, Michael and Sycara, Katia , booktitle =. 2023 , month = dec, address =. doi:10.18653/v1/2023.emnlp-main.13 , url = on Pithdoi:10.18653/v1/2023.emnlp-main.13
- [31] Qiu, Shuwen and Liu, Mingdian and Li, Hengli and Zhu, Song-Chun and Zheng, Zilong , booktitle =. 2024 , month = sep, address =. doi:10.18653/v1/2024.sigdial-1.63 , url = on Pithdoi:10.18653/v1/2024.sigdial-1.63
- [32] Yang, Bo and Guo, Jiaxian and Iwasawa, Yusuke and Matsuo, Yutaka , journal =. 2025 , doi = on Pith
- [33] Mu, Chunjiang and Zeng, Ya and Zhang, Qiaosheng and Shao, Kun and Chu, Chen and Guo, Hao and Jia, Danyang and Wang, Zhen and Hu, Shuyue , journal =. 2026 , month = mar, doi = on Pith
- [34] Hwang, EunJeong and Yin, Yuwei and Carenini, Giuseppe and West, Peter and Shwartz, Vered , booktitle =. 2026 , month = jul, address =. doi:10.18653/v1/2026.findings-acl.551 , url = on Pithdoi:10.18653/v1/2026.findings-acl.551
- [35] Almeida, Diogo , year = on Pith
- [36] Hu, Jianyi and Zhang, Hangtao and Liu, Yi and Zeng, Yeqi and Zeng, Li and Wang, Xianlong and Wang, Rui and Zhang, Leo Yu , year =. arXiv preprint arXiv:2609.31142 , url = on PithPith's reviewarXiv
- [37] Chan, Chunkit and Yim, Yauwai and Zeng, Hongchuan and Zou, Zhiying and Cheng, Xinyuan and Sun, Zhifan and Deng, Zheye and Chung, Kawai and Ao, Yuzhuo and Fan, Yixiang and Jiayang, Cheng and Nie, Ercong and Wong, Ginny Y. and Schmid, Helmut and Sch \"u tze, Hinrich and See, Sim... on Pith
- [38] Cohen, Jacob , title =. Educational and Psychological Measurement , year = on Pith
- [39] Fleiss, Joseph L. , title =. Psychological Bulletin , year = on Pith
- [40] Chung, Kawai and Chan, Chunkit and Yim, Yauwai and Liu, Yuxuan and Shi, Haochen and Wang, Weiqi and Zong, Qing and Zheng, Tianshi and Fu, Yixuan and Wong, Kai Chung and Liang, Hao and Gao, Yifan and Yang, Xi and Hsiao, Janet Hui-wen and Song, Yangqiu , year =. arXiv preprint a... on PithPith's reviewarXiv
- [41] Rao, Delip and Callison-Burch, Chris , year =. arXiv preprint arXiv:2609.29769 , note = on PithPith's reviewarXiv
- [42] 2024 , month = jul, url = on Pith
- [43] 2025 , month = apr, url = on Pith
- [44] 2023 , month = nov, url = on Pith
- [45] 2026 , month = jul, url = on Pith
- [46] Grattafiori, Aaron and others , year =. arXiv preprint arXiv:2407.21783 , url = on PithPith's reviewarXiv
- [47] Bakouch, Elie and Ben Allal, Loubna and Lozhkov, Anton and Tazi, Nouamane and Tunstall, Lewis and Pati. 2025 , url = on Pith
- [48] 2026 , howpublished = on Pith
- [49] Sclar, Melanie and Yu, Jane and Fazel-Zarandi, Maryam and Tsvetkov, Yulia and Bisk, Yonatan and Choi, Yejin and Celikyilmaz, Asli , booktitle =. 2025 , url = on Pith
- [50] Jung, Chani and Kim, Dongkwan and Jin, Jiho and Kim, Jiseon and Seonwoo, Yeon and Choi, Yejin and Oh, Alice and Kim, Hyunwoo , booktitle =. 2024 , pages =. doi:10.18653/v1/2024.emnlp-main.1105 , url = on Pithdoi:10.18653/v1/2024.emnlp-main.1105
- [51] Jin, Chuanyang and Wu, Yutong and Cao, Jing and Xiang, Jiannan and Kuo, Yen-Ling and Hu, Zhiting and Ullman, Tomer and Torralba, Antonio and Tenenbaum, Joshua B. and Shu, Tianmin , journal =. 2024 , url = on Pith
- [52] Shi, Haojun and Ye, Suyu and Fang, Xinyu and Jin, Chuanyang and Isik, Leyla and Kuo, Yen-Ling and Shu, Tianmin , journal =. 2024 , url = on Pith
- [53] Turpin, Miles and Michael, Julian and Perez, Ethan and Bowman, Samuel R. , booktitle =. 2023 , url = on Pith
- [54] Lanham, Tamera and Chen, Anna and Radhakrishnan, Ansh and Steiner, Benoit and Denison, Carson and Hernandez, Danny and Li, Dustin and Durmus, Esin and Hubinger, Evan and Kernion, Jackson and Lukošiūtė, Kamilė and Nguyen, Karina and Cheng, Newton and Joseph, Nicholas and Schief... on Pith
- [55] Yao, Shunyu and Yu, Dian and Zhao, Jeffrey and Shafran, Izhak and Griffiths, Tom and Cao, Yuan and Narasimhan, Karthik , booktitle =. 2023 , doi = on Pith
- [56] Ren, Allen Z. and Dixit, Anushri and Bodrova, Alexandra and Singh, Sumeet and Tu, Stephen and Brown, Noah and Xu, Peng and Takayama, Leila and Xia, Fei and Varley, Jake and Xu, Zhenjia and Sadigh, Dorsa and Zeng, Andy and Majumdar, Anirudha , booktitle =. 2023 , publisher = on Pith
Cited by 2 Pith papers
Reviewed papers on Pith that cite this one. The full citation record
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment