LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue

Katharine Kowalyshyn; Matthias Scheutz

REVIEW 4 major objections 5 minor 1 cited by

LLMs can annotate beliefs and goals in team dialogue, but systematically err where spatial reasoning or prosodic cues carry the meaning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

LLMs can imitate mental-state annotation in team dialogue but systematically err on spatial reasoning and prosodic cues, per a six-dialogue CReST pilot.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Useful SMM framework and a genuinely new ToM probe, but the cross-model comparison is confounded by the same model acting as annotator and judge. the 4 major comments →

arxiv 2509.02292 v2 pith:RZHS77OV submitted 2025-09-02 cs.CL

LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue

Katharine Kowalyshyn , Matthias Scheutz This is my paper

classification cs.CL

keywords theory of mindlarge language modelsshared mental modelssituated dialoguebelief-state annotationCReST corpusteam coherenceprosody

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that large language models can be pressed into service as annotators of a team's shared mental model — labeling, utterance by utterance, what each teammate believes, is committed to, and is trying to achieve — and as automatic detectors of when those mental states diverge. Working from six transcribed search-style dialogues in which a remote director guides a physically present searcher through a building, the authors compare naive human, LLM, and video-grounded gold annotations. They find that LLMs look coherent on plain-language annotation but systematically miss the mark where meaning hangs on spatial layout or on how an utterance was said. If the finding holds, an LLM cannot yet be trusted to watch a human team's shared understanding drift and warn them before coordination breaks down — the failure mode sits exactly where human teamwork is hardest.

Core claim

The paper's central claim is that LLMs can track, utterance by utterance in transcribed team dialogue, what each member believes, is committed to, and is trying to achieve — but that the competence is shallow. Across six episodes of the Cooperative Remote Search Task, three LLMs, two naive humans, and two video-informed humans annotated every utterance with belief, goal, and commitment states; a second LLM compared each naive annotation against the video-grounded gold standard and classified divergences as belief contradictions, false beliefs, unsupported beliefs, or omissions. LLM annotations prove fluent and internally consistent, yet diverge from ground truth precisely where meaning depen

What carries the argument

The load-bearing object is the annotation schema: nine fields per utterance — what the searcher believes, what the director believes, both second-order beliefs, both agents' commitments, both agents' goals, and the common belief — which forces any annotator to commit to a concrete mental-state update at every dialogue move. Around it sits a four-type discrepancy taxonomy (belief contradiction, false belief, unsupported belief, omission) with a severity ordering and a weighted, per-utterance, min-max-normalized coherence score that puts any two mental annotations on a fixed [0,1] scale. The two-step pipeline — annotation, then discrepancy detection by a second LLM — is what turns raw dialogue

Load-bearing premise

The entire cross-model comparison treats the LLM that counts discrepancies as a neutral measuring stick, equally fair to every annotator it scores, even though the paper's own validation shows that detector's accuracy wobbles between 52% and 88% and is near chance for the human annotations.

What would settle it

Re-score the six dialogues' annotations with one fixed detector — human judges or a single held-out LLM counting discrepancies for every annotator — and check whether the ranking survives. If o3-mini no longer looks cleanest when its annotations are graded by Gemma or a human judge rather than by itself, the cross-model comparison is a self-grading artifact. The same test also shows whether the spatial and prosodic failures belong to the annotation task itself or to whichever model happens to be scoring it.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

LLMs cannot be trusted as standalone monitors of a team's shared mental model: the paper concludes hybrid LLM-plus-human validation is needed before such annotations inform decisions.
The discrepancy framework offers a reusable, severity-weighted metric for comparing any two mental annotations, human or machine, not just LLM versus gold.
Situated dialogue annotation is proposed as a more ecologically valid Theory-of-Mind benchmark for AI than static multiple-choice or story-comprehension tests.
Models fail in different directions — one over-speculates, another omits, another over-commits — so a single coherence score can mask which error mode is dangerous for a given task.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Because each model graded its own annotations, the reported ordering (o3-mini seeming cleanest, Claude noisiest) may be a self-grading artifact; scoring every annotator with one fixed, human-validated detector could reshuffle the ranking — the paper's own Dialogue-1 validation shows detector accuracy spanning 52–88%.
The min-max normalization guarantees some annotator-dialogue pair receives 1.0 and another 0.0, so the coherence scores are relative within this six-dialogue set, not absolute quality measurements.
Giving the annotating LLM the same partial map the director holds would isolate whether the spatial errors come from missing information or from a genuine deficit in spatial reasoning — an experiment the paper motivates but does not run.
Run in real time, the discrepancy taxonomy could flag the moment a team's shared understanding starts to drift; the paper motivates this intervention use case but tests only post-hoc annotation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Useful SMM framework and a genuinely new ToM probe, but the cross-model comparison is confounded by the same model acting as annotator and judge.

read the letter

Short version: the paper has a useful idea and a real confound. The authors are the first to use belief-state annotations over situated team dialogue as a ToM probe, and the SMM coherence framework with its four discrepancy types is a sensible way to think about annotation divergence. The manual validation on Dialogue 1 is honest work. But the central quantitative comparison across models is not identifiable: the same model that annotated is also the discrepancy detector, and the validation shows detector accuracy differs by model (52–88%). So Claude Sonnet 4 looking "overgeneralizing" may just be its detector being willing to flag discrepancies, and o3-mini looking clean may be its detector missing things. That is not a small caveat; it is the load-bearing joint.

Also, the abstract's claim about spatial reasoning and prosodic disambiguation never gets a direct test. No table or analysis breaks errors down by cause. That claim should either be demonstrated or removed.

What is genuinely useful: the task framing, the annotation schema (beliefs, goals, commitments, second-order beliefs), and the discrepancy taxonomy. The human baseline is a good addition. The D1 hand-check is a step in the right direction but partial. The sample of six dialogues is small but appropriate for a pilot; that is not the problem. The absence of released code/data is annoying, though they promise it later.

Recommendation: I would send this to peer review, not desk reject. A competent referee could push them to use a single neutral discrepancy detector or hand-validate all detected discrepancies, and to do a per-cause error analysis. The framework is worth preserving; the empirical claims need to be rebuilt. If I were the editor, I'd ask for major revision.

Referee Report

4 major / 5 minor

Summary. The paper proposes a two-step framework for evaluating Theory of Mind (ToM)-like reasoning in LLMs using situated team dialogue from the CReST corpus. Three LLMs (o3-mini, Claude Sonnet 4, Gemma 8.5B), a pair of naive humans, and a ground-truth annotator team label beliefs, goals, and commitments for six dialogues. A secondary LLM then compares naive annotations against ground truth and classifies discrepancies into four types: belief contradictions, false beliefs, unsupported beliefs, and omissions. From these counts the authors define a weighted per-utterance discrepancy score and a normalized coherence score. The paper reports that Claude Sonnet 4 produces the most discrepancies, o3-mini and Gemma produce fewer, humans produce the fewest unsupported beliefs, and LLMs fail in spatial reasoning and prosodic disambiguation. The main deliverables are the annotation dataset, the discrepancy framework, and the empirical comparison.

Significance. If the empirical results were reliable, the paper would make a useful contribution by introducing a situated-dialogue benchmark for LLM ToM evaluation and a reusable discrepancy-analysis framework. The use of an external human ground truth, the detailed prompt documentation in Appendix A, and the explicit human validation of Dialogue 1 in Appendix C are commendable and represent more than the typical LLM-as-judge setup. However, the central quantitative conclusions are undermined by a same-model-as-annotator-and-detector confound, and the headline spatial/prosody claim is never operationalized. The paper's value is therefore currently more methodological than evidentiary; a substantial reanalysis is needed before the empirical rankings can be trusted.

major comments (4)

[§5, Table 2 footnote; Appendix C, Table 4] The central comparison is confounded. The Table 2 footnote states that the same LLM model was used as discrepancy identifier for its own annotations, and human annotations were scored using Claude Sonnet 4. Appendix C reports manual validation on Dialogue 1 with detector accuracy varying by annotator: o3-mini 0.876, Gemma 0.791, Claude Sonnet 4 0.725, and naive-human/Claude 0.524. Thus the discrepancy counts in Table 2 are not model-neutral measurements of annotation quality. A detector that is more willing to classify discrepancies will inflate its own annotator's totals, while a detector that misses discrepancies will deflate them. The observed pattern—Claude Sonnet 4 having the highest totals and humans having very few unsupported beliefs—cannot be separated from detector sensitivity. The cross-model conclusions in §5.1 and §5.2 are therefore not identifiable from these data. A single
[Abstract; §5.1; §6] The abstract's central claim that LLMs 'systematically err in scenarios requiring spatial reasoning or disambiguation of prosodic cues' is never operationalized. Table 2 and Appendix E categorize discrepancies only as belief contradictions, false beliefs, unsupported beliefs, and omissions; no table or analysis decomposes errors by spatial vs. non-spatial or by prosodic cause. Moreover, the LLM input is the transcribed text (Appendix A), so prosodic cues are not available to the models; the corpus audio is not used in the annotation prompts or the discrepancy prompts. The discussion's reference to 'increased spatial ambiguity' for Dialogues 3 and 5 is an untested speculation, not a result. The paper should either add an error-cause analysis tied to spatial/prosodic factors or remove this claim from the abstract.
[Appendix C; §5] The validation supporting detector reliability covers only Dialogue 1 and reports only overall accuracy. Table 2 aggregates over six dialogues and four annotators, so Dialogue 1 accuracy (0.524–0.876) does not establish that the counts are trustworthy in the other five dialogues. Accuracy alone is also insufficient: a detector can be accurate overall while systematically biasing a particular discrepancy type, and precision/recall are not reported. The paper proceeds as if the detector counts are ground truth. A per-dialogue validation or calibration is needed before any comparative rankings are presented.
[§4.1.3, §4.2; Table 2] The discrepancy categories are not mutually exclusive. Section 4.1.3 states that unsupported beliefs 'may be false or a belief contradiction, as well as combinations thereof,' and the discrepancy detection prompt in Appendix A instructs the model to classify each discrepancy 'into one or more of the types above.' Yet Table 2 reports each type as a separate count, and no aggregation or priority rule is given. A single discrepancy could be counted in multiple rows, inflating totals. The paper should either define mutually exclusive categories or provide a deterministic multi-label counting rule before totals are compared.

minor comments (5)

[Appendix A] The discrepancy detection prompt says 'using the following five types' but enumerates only four (belief contradiction, false belief, omission, unsupported belief). Fix the count or add the missing type.
[Table 2] The footnote refers to blue and purple highlighting, but the table as rendered is monochrome. Use accessible symbols or annotation to indicate the min/max values.
[§5.1] The claim that Dialogues 3 and 5 'consistently elicit higher discrepancy counts' is not supported by the per-utterance rates in Appendix E. For example, unsupported beliefs per utterance are highest in D4 for Claude Sonnet 4 (1.617), not D3 or D5, and omissions per utterance are also high in D4 for o3-mini and Claude. The claim should be checked against per-utterance rather than raw counts.
[§3.1] The text first says the ground-truth annotations were made by 'two humans' and then says they were 'corroborated by three humans.' Clarify the adjudication process and the number of annotators.
[Eq. (3)] The normalized score S_m,d is relative because it is normalized by the global minimum and maximum over all (m,d) pairs. A score of 1.000 or 0.000 is therefore an artifact of the particular corpus and set of annotators, not an absolute property. Reporting raw per-utterance rates (Appendix E) is preferable; the normalized score should be described as corpus-relative.

Circularity Check

1 steps flagged

Cross-model discrepancy rankings are self-referential: each annotator is scored by its own LLM detector, and Appendix C shows detector sensitivity varies from 52% to 88%.

specific steps

other [Section 5, Table 2 footnote; Appendix C, Table 4]
"The same LLM model was run as a discrepancy identifier as its annotator and human naive annotations were run using Claude Sonnet 4."

The central quantitative output (Tables 2/3) is produced by a discrepancy detector that, for each LLM, is the same model that produced the annotations (human annotations scored by Claude). Appendix C's own validation shows detector agreement varies: o3-mini 0.876, Gemma 0.791, Claude 0.725, naive-human/Claude 0.524. Thus the reported differences between annotators are confounded with detector sensitivity. Claude's high totals and humans' low unsupported-belief counts may reflect the judge rather than the annotator. The conclusions that Claude 'overgeneralizes' and humans 'limit unsupported beliefs' reduce, in part, to the self-evaluation setup rather than externally validated annotation content.

full rationale

The paper's framework itself is not circular: the gold-standard annotations come from humans with video access, and the discrepancy categories are defined independently. However, the implemented comparison in Tables 2 and 3 uses a different, model-specific judge for each annotator, and the judges are the same models whose annotations they grade. Appendix C validates only Dialogue 1 and finds substantial variation in judge accuracy, including near-chance performance (52.4%) for the human/Claude pair, yet the authors 'proceeded with using our method' and report cross-model rankings as if the counts were model-neutral. That is the load-bearing step: the empirical rankings and the abstract's systematic-error conclusions are not identifiable separately from the judge's identity. No other circularity was found: the few self-citations (e.g., Scheutz et al. 2017, 2024) provide background or prior team-coherence results and are not used to justify the present derivation; there is no imported uniqueness theorem or fitted parameter renamed as a prediction. The severity metric and normalization are explicit and non-circular, though normalization by global min/max makes scores relative. The main non-circular contribution is the annotated dataset and the framework; the cross-model empirical assessment is partially self-referential because of the judge confound.

Axiom & Free-Parameter Ledger

1 free parameters · 3 axioms · 1 invented entities

The paper rests on domain assumptions about the validity of the annotation scheme and gold standard, plus the ad hoc assumption that LLM discrepancy detectors are reliable. The only free parameter is the weighting, set to 1 for all types. The invented entity is the coherence score, a dataset-relative metric with no external benchmark.

free parameters (1)

Discrepancy weights wx = 1 for all four types
The severity ranking in Eq. 1 is made concrete by setting all four weights to 1. The paper acknowledges weights are at the analyst's discretion, so absolute scores are not intrinsic.

axioms (3)

domain assumption The video-derived annotations are treated as ground truth for mental states.
Section 3.1 uses headcam video to establish ground truth, but video provides environmental facts, not direct access to beliefs or goals.
domain assumption The belief/goal/commitment annotation scheme captures shared mental models.
The prompt in Appendix A defines eight fields, assuming these are the relevant SMM components; no external validation is given.
ad hoc to paper The four discrepancy types are exhaustive and can be reliably detected by an LLM prompt.
The paper defines these categories and relies on an LLM to detect them, but D1 validation shows detector accuracy varies by model (52-88%), so this assumption is only partially checked.

invented entities (1)

Weighted SMM coherence score Sm,d no independent evidence
purpose: A single number to compare annotation quality across models and dialogues.
Introduced in Eq. 3 via min-max normalization over the current dataset; no external validation or theoretical derivation, and the score changes if new data are added.

reviewed 2026-08-05 · how reviews work

0 comments

Cite this review

Pith. "Pith review of LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue." pith.science (2026). https://pith.science/paper/RZHS77OV

@misc{pith2026250902292,
  author       = {Pith},
  title        = {Pith review of: LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RZHS77OV}},
  note         = {Machine review of arXiv:2509.02292}
}

Share X Bluesky LinkedIn Reddit HN

read the original abstract

What if large language models could not only infer human mindsets but also expose every blind spot in team dialogue such as discrepancies in the team members' joint understanding? We present a novel, two-step framework that leverages large language models (LLMs) both as human-style annotators of team dialogues to track the team's shared mental models (SMMs) and as automated discrepancy detectors among individuals' mental states. In the first step, an LLM generates annotations by identifying SMM elements within task-oriented dialogues from the Cooperative Remote Search Task (CReST) corpus. Then, a secondary LLM compares these LLM-derived annotations and human annotations against gold-standard labels to detect and characterize divergences. We define an SMM coherence evaluation framework for this use case and apply it to six CReST dialogues, ultimately producing: (1) a dataset of human and LLM annotations; (2) a reproducible evaluation framework for SMM coherence; and (3) an empirical assessment of LLM-based discrepancy detection. Our results reveal that, although LLMs exhibit apparent coherence on straightforward natural-language annotation tasks, they systematically err in scenarios requiring spatial reasoning or disambiguation of prosodic cues.

Figures

Figures reproduced from arXiv: 2509.02292 by Katharine Kowalyshyn, Matthias Scheutz.

**Figure 2.** Figure 2: Using an example from Dialogue 1, we show [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Figure 3: An overview of our two-step process used to annotate and find discrepancies between SMMs using LLMs. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

**Figure 4.** Figure 4: Visualization of the total discrepancies iden [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Not Another EHR: Reimagining Physician Information Needs with Generative AI Technology
cs.HC 2026-03 unverdicted novelty 4.0

Generative AI can enable dynamic, adaptive interfaces that better support physicians' information needs in navigating and synthesizing electronic health record data.

Reference graph

Works this paper leans on

19 extracted references · 13 canonical work pages · cited by 1 Pith paper

[1]

Searcher: The searcher is in a remotely located real world environment of a floor in a building with many rooms, boxes, and doors

work page
[2]

speaker": <speaker>,

Director: The director is local but has a static (but somewhat flawed) map of the floor showing some of the boxes, rooms and doors, and is communicating and coordinating via audio with the searcher. YOUR TASK: You will carefully listen to the conversation (current dialogue move and recent dialogue history) and logically deduce/induce/abduce beliefs, goals...

work page
[3]

Identify the current state of the world

work page
[4]

The [searcher/director] believes

Based on the current dialogue move and the history, describe what beliefs, goals, and commitments of the director and searcher you can infer (implicitly or explicitly). Use ONLY the phrasing "The [searcher/director] believes", "The [searcher/director] is committed to", "The [searcher/director] 's goal is" to begin the annotation

work page
[5]

no change

Reason through this and rationalize why you included the updates that you chose to include. If no updates are needed, write "no change" for every field in the Annotation object. Output MUST be a JSON string, and NOT markdown. IMPORTANT: Output ONLY the JSON object, with no explanation or commentary. Do not include any reasoning or markdown formatting. 12 ...

work page
[6]

In 2024 IEEE Intelligent Vehicles Symposium (IV) , pages 1206–1213

Tr2mtl: Llm based framework for metric tem- poral logic formalization of traffic rules. In 2024 IEEE Intelligent Vehicles Symposium (IV) , pages 1206–1213. John E Mathieu, Tonia S Heffner, Gerald F Goodwin, Eduardo Salas, and Janis A Cannon-Bowers. 2000. The influence of shared mental models on team pro- cess and performance. Journal of applied psychology...

Pith/arXiv arXiv 2024
[8]

arXiv preprint arXiv:2403.18120

Don’t trust: Verify–grounding llm quantita- tive reasoning with autoformalization. arXiv preprint arXiv:2403.18120. 10 A Full Prompts The following section outlines our prompts used for the experiments. These prompts were all given the exact same way to all models to compare models as effectively as possible. To note, we tried many different prompts durin...

Pith/arXiv arXiv
[11]

Identify the speaker of the current dialogue move

work page
[12]

Could be one of question | assertion/statement | command/request | offer | promise | acknowledgement | greeting

Identify the dialogue act of the move. Could be one of question | assertion/statement | command/request | offer | promise | acknowledgement | greeting. This can influence what you conclude about the beliefs. For example, questions about p suggest that the speaker did not previously believe p

work page
[16]

Belief Contradiction: One annotator identifies a belief b, and the other identifies not b

work page
[17]

False Belief: An annotator believes something that contradicts the known ground truth

work page
[18]

Omission: The ground truth includes a belief that the annotator simply omits

work page
[19]

Discrepancies

Unsupported Belief: A belief not verifiable from the context or ground truth; lacks support either way. For each discrepancy: - Classify it into one or more of the types above. - Clearly state what each agent believes, using plain English. - Provide a short explanation. Output Format: You must output a single JSON object with the following structure. For ...

work page
[2010]

cooperative remote search task

The indiana" cooperative remote search task"(crest) corpus. In LREC. Gwendolyn Edgar, Ayca Aygun, Matthew McWilliams, and Matthias Scheutz. 2024. Toward genuine robot teammates: Improving human-robot team perfor- mance beyond shared mental models with proactivity. In Discovering the Frontiers of Human-Robot Inter- action: Insights and Innovations in Colla...

work page 2024
[2012]

Linguis- tic Issues in Language Technology, 7

Parallel syntactic annotation in crest. Linguis- tic Issues in Language Technology, 7. Jiawen Liu, Yuanyuan Yao, Pengcheng An, and Qi Wang. 2024a. Peergpt: Probing the roles of llm- based peer agents as team moderators and partici- pants in children’s collaborative learning. Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, p...

Pith/arXiv arXiv
[2016]

Frontiers in Robotics and AI, 3:62

Team communication as a collaborative pro- cess. Frontiers in Robotics and AI, 3:62. P.N. Johnson-Laird. 1980. Mental models in cognitive science. Cognitive Science, 4(1):71–115. Catholijn M Jonker, M Birna Van Riemsdijk, and Bas Vermeulen. 2010. Shared mental models: A concep- tual analysis. In International workshop on coordina- tion, organizations, ins...

Pith/arXiv arXiv 1980
[2017]

Jour- nal of Cognitive Engineering and Decision Making, 11(3):203–224

A framework for developing and using shared mental models in human-agent teams. Jour- nal of Cognitive Engineering and Decision Making, 11(3):203–224. Matthias Scheutz, Bradley Oosterveld, John Peterson, Eric Wyss, and Evan Krause. 2024. A multi-robot architecture framework for effective robot teammates in mixed-initiative teams. In Proceedings of the 202...

Pith/arXiv arXiv 2024
[2023]

In Findings of the Association for Computational Linguistics: EMNLP 2023

Towards a holistic landscape of situated theory of mind in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023. Kumar Manas, Stefan Zwicklbauer, and Adrian Paschke

work page 2023
[2024]

Preprint, arXiv:2402.15052

Tombench: Benchmarking theory of mind in large language models. Preprint, arXiv:2402.15052. Kathleen M Eberhard, Hannele Nicholson, Sandra Kübler, Susan Gundersen, and Matthias Scheutz

Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.

[1] [1]

Searcher: The searcher is in a remotely located real world environment of a floor in a building with many rooms, boxes, and doors

work page

[2] [2]

speaker": <speaker>,

Director: The director is local but has a static (but somewhat flawed) map of the floor showing some of the boxes, rooms and doors, and is communicating and coordinating via audio with the searcher. YOUR TASK: You will carefully listen to the conversation (current dialogue move and recent dialogue history) and logically deduce/induce/abduce beliefs, goals...

work page

[3] [3]

Identify the current state of the world

work page

[4] [4]

The [searcher/director] believes

Based on the current dialogue move and the history, describe what beliefs, goals, and commitments of the director and searcher you can infer (implicitly or explicitly). Use ONLY the phrasing "The [searcher/director] believes", "The [searcher/director] is committed to", "The [searcher/director] 's goal is" to begin the annotation

work page

[5] [5]

no change

Reason through this and rationalize why you included the updates that you chose to include. If no updates are needed, write "no change" for every field in the Annotation object. Output MUST be a JSON string, and NOT markdown. IMPORTANT: Output ONLY the JSON object, with no explanation or commentary. Do not include any reasoning or markdown formatting. 12 ...

work page

[6] [6]

In 2024 IEEE Intelligent Vehicles Symposium (IV) , pages 1206–1213

Tr2mtl: Llm based framework for metric tem- poral logic formalization of traffic rules. In 2024 IEEE Intelligent Vehicles Symposium (IV) , pages 1206–1213. John E Mathieu, Tonia S Heffner, Gerald F Goodwin, Eduardo Salas, and Janis A Cannon-Bowers. 2000. The influence of shared mental models on team pro- cess and performance. Journal of applied psychology...

Pith/arXiv arXiv 2024

[7] [8]

arXiv preprint arXiv:2403.18120

Don’t trust: Verify–grounding llm quantita- tive reasoning with autoformalization. arXiv preprint arXiv:2403.18120. 10 A Full Prompts The following section outlines our prompts used for the experiments. These prompts were all given the exact same way to all models to compare models as effectively as possible. To note, we tried many different prompts durin...

Pith/arXiv arXiv

[8] [11]

Identify the speaker of the current dialogue move

work page

[9] [12]

Could be one of question | assertion/statement | command/request | offer | promise | acknowledgement | greeting

Identify the dialogue act of the move. Could be one of question | assertion/statement | command/request | offer | promise | acknowledgement | greeting. This can influence what you conclude about the beliefs. For example, questions about p suggest that the speaker did not previously believe p

work page

[10] [16]

Belief Contradiction: One annotator identifies a belief b, and the other identifies not b

work page

[11] [17]

False Belief: An annotator believes something that contradicts the known ground truth

work page

[12] [18]

Omission: The ground truth includes a belief that the annotator simply omits

work page

[13] [19]

Discrepancies

Unsupported Belief: A belief not verifiable from the context or ground truth; lacks support either way. For each discrepancy: - Classify it into one or more of the types above. - Clearly state what each agent believes, using plain English. - Provide a short explanation. Output Format: You must output a single JSON object with the following structure. For ...

work page

[14] [2010]

cooperative remote search task

The indiana" cooperative remote search task"(crest) corpus. In LREC. Gwendolyn Edgar, Ayca Aygun, Matthew McWilliams, and Matthias Scheutz. 2024. Toward genuine robot teammates: Improving human-robot team perfor- mance beyond shared mental models with proactivity. In Discovering the Frontiers of Human-Robot Inter- action: Insights and Innovations in Colla...

work page 2024

[15] [2012]

Linguis- tic Issues in Language Technology, 7

Parallel syntactic annotation in crest. Linguis- tic Issues in Language Technology, 7. Jiawen Liu, Yuanyuan Yao, Pengcheng An, and Qi Wang. 2024a. Peergpt: Probing the roles of llm- based peer agents as team moderators and partici- pants in children’s collaborative learning. Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, p...

Pith/arXiv arXiv

[16] [2016]

Frontiers in Robotics and AI, 3:62

Team communication as a collaborative pro- cess. Frontiers in Robotics and AI, 3:62. P.N. Johnson-Laird. 1980. Mental models in cognitive science. Cognitive Science, 4(1):71–115. Catholijn M Jonker, M Birna Van Riemsdijk, and Bas Vermeulen. 2010. Shared mental models: A concep- tual analysis. In International workshop on coordina- tion, organizations, ins...

Pith/arXiv arXiv 1980

[17] [2017]

Jour- nal of Cognitive Engineering and Decision Making, 11(3):203–224

A framework for developing and using shared mental models in human-agent teams. Jour- nal of Cognitive Engineering and Decision Making, 11(3):203–224. Matthias Scheutz, Bradley Oosterveld, John Peterson, Eric Wyss, and Evan Krause. 2024. A multi-robot architecture framework for effective robot teammates in mixed-initiative teams. In Proceedings of the 202...

Pith/arXiv arXiv 2024

[18] [2023]

In Findings of the Association for Computational Linguistics: EMNLP 2023

Towards a holistic landscape of situated theory of mind in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023. Kumar Manas, Stefan Zwicklbauer, and Adrian Paschke

work page 2023

[19] [2024]

Preprint, arXiv:2402.15052

Tombench: Benchmarking theory of mind in large language models. Preprint, arXiv:2402.15052. Kathleen M Eberhard, Hannele Nicholson, Sandra Kübler, Susan Gundersen, and Matthias Scheutz

Pith/arXiv arXiv

LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue

The reading

Core claim §

What carries the argument

Load-bearing premise §

What would settle it §

If this is right

Where Pith is reading between the lines

discussion (0)

Core claim

Load-bearing premise

What would settle it