Pith. sign in

REVIEW 4 major objections 5 minor 37 references

Multimodal AI models can identify locally plausible motivations but nearly never keep them consistent across a whole story.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 15:16 UTC pith:IVF2QOB7

load-bearing objection Builds a genuinely new benchmark for sequential motivation reasoning, but the human ceiling is entangled with the AI-draft pipeline and the context-ablation result cuts against the central narrative—worth refereeing, needs fixes. the 4 major comments →

arxiv 2607.26465 v1 pith:IVF2QOB7 submitted 2026-07-29 cs.AI

MultivationBench: A Benchmark for Multimodal Sequential Motivation Reasoning

Kawai Chung , Chunkit Chan , Yauwai Yim , Yuxuan Liu , Haochen Shi , Weiqi Wang , Qing Zong , Tianshi Zheng
show 7 more authors
Yixuan Fu Kai Chung Wong Hao Liang Yifan Gao Xi Yang Janet Hui-wen Hsiao Yangqiu Song
This is my paper
classification cs.AI
keywords multimodal large language modelsmotivation reasoningsequential reasoningbenchmarkMaslow hierarchy of needsReiss basic desiresstory-level consistencysocial intelligence
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

MULTIVATIONBENCH is a benchmark for testing whether multimodal large language models can do sequential motivation reasoning: inferring why a character acts, then revising that inference as later images and text accumulate. Built around two established psychological taxonomies, it asks four multi-label questions per visually grounded behavior across 1,000 stories. The paper's main finding is a measured gap: the best model reaches only 39.2% exact-match accuracy and 55.0% F1, while full story-level consistency is 0.80%, and human annotators score 60.7–78.6% exact match. The authors interpret the gap as evidence that current models can recognize a plausible motivation from a single moment but do not engage in the dynamic, cumulative reasoning that character-level understanding requires.

Core claim

The paper's central discovery is that current multimodal large language models show a measurable split between static and sequential motivation reasoning. On MULTIVATIONBENCH's 16,092 questions, the strongest overall exact match is 39.2% and the strongest F1 is 55.0%, but a story counts as fully consistent only if every question in its sequence is answered exactly, and the best full-story consistency is 0.80%. Humans reach 60.7–78.6% exact match across the four tasks. Error analysis shows why: models favor over-interpretation, adding motivations that later evidence does not support, and per-label performance is highest for cues readable in a single frame and lowest for motives that require a

What carries the argument

MULTIVATIONBENCH itself is the central instrument. Each of its 1,000 story-driven visual narratives contributes a behavior chain — a list of visually grounded actions mapped to image indices — and each behavior gets four multi-label questions: Maslow Definition, Maslow Practical Motivation, and the same pair for Reiss's 16 basic desires. The definition tasks test category knowledge; the practical tasks test situated inference. The story-level consistency score is the key mechanism: because it requires every question in a story to be answered exactly right, it turns the benchmark from an accuracy measurement into a test of whether a model can revise and sustain a single interpretation across

Load-bearing premise

The benchmark's gold labels, drafted by AI and ratified by three annotators, are treated as valid ground truth for which motivations a narrative supports.

What would settle it

Take the paper's 150-behavior context-ablation sample and administer the same reduced-context questions to human raters. If humans, like the models, show almost no performance drop when earlier context is removed, the claim that models 'fail to revise' collapses; if humans drop sharply, the earlier context is load-bearing and the model behavior is a genuine deficit.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the benchmark measures what it claims, the current generation of multimodal models lacks a capability central to social intelligence: revising an earlier motivational interpretation when new visual and textual evidence arrives.
  • Models' F1 scores survive text-only input better than exact match, so text carries much of the gist, but visual evidence is needed to disambiguate the exact set of motivations.
  • Fine-grained taxonomies are the harder test: performance drops from Maslow's 8 categories to Reiss's 16, indicating that distinguishing closely related desires is a separate bottleneck.
  • The 0.80% full-story consistency score means that even the best model almost never sustains a correct interpretation through an entire narrative, so per-question accuracy is not translating into narrative-level coherence.
  • Longer narratives (over roughly 13 images) degrade performance, suggesting the difficulty is cumulative evidence integration, not just context length.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The passive benchmark could be turned into an active one — letting the model request the next frame or act on its interpretation would test whether revision failure is a measurement artifact or a real capability gap.
  • A human-control version of the paper's context-ablation experiment would separate failure to use context from failure to revise: if humans also show no performance drop when earlier context is removed, the 'revision deficit' needs reinterpreting.
  • The authors leave untested whether an explicit revision prompt — asking the model to reconsider an earlier answer after seeing new frames — improves consistency; the benchmark is well suited to measure such prompting.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces MULTIVATIONBENCH, a benchmark of 1,000 story-driven visual narratives (drawn from SSID, StoryReasoning, and MovieBench) with 4,023 behavior instances and four tasks each: Maslow Definition, Maslow Practical Motivation, Reiss Definition, and Reiss Practical Motivation. The tasks are multi-label and grounded in Maslow's expanded hierarchy and Reiss's 16 basic desires. The paper evaluates eight MLLMs (closed- and open-source) in multimodal, text-only, and image-only settings, reporting instance-level EM/F1, story-level consistency, and various ablations. The central claim is that all tested models perform far below human annotators (best model EM 39.2% vs. human 60.7–78.6%; story-level full consistency at most 0.80%), and that this gap reflects a specific failure of sequential motivation reasoning rather than merely static recognition.

Significance. If the benchmark is valid, it addresses a genuine gap: existing motivation-reasoning benchmarks are mostly static and text-only, whereas humans infer and revise motivations as multimodal evidence accumulates. The paper's strengths include a relatively large constructed dataset, multiple complementary tasks, a broad set of evaluated model families, explicit contamination probing, same-behavior context ablations, and careful reporting of soft story-level metrics. The empirical pattern—models can partially match labels but rarely maintain consistent full-story correctness—is potentially valuable for the community. However, the validity of the headline claim depends on the gold labels and the human ceiling being independent, stable standards. The construction pipeline and the human evaluation design raise concerns that the measured 'critical disconnect' may be partly an artifact of AI-seeded interpretation and shared annotation procedures.

major comments (4)
  1. [§3.3, Fig. 2, Appendix D] The gold labels are not human-originated: Grok-4.1 and Gemini-3 draft behavior chains, candidate motivations, and practical options, and the three graduate-student annotators review/ratify them (§3.3, Fig. 2, Appendix D.1–D.2). Because the human ceiling (Appendix E.4) is measured on these same three annotators via majority vote, both the gold standard and the 'human' reference share the AI-drafted interpretive scheme. The negative control in Appendix B.2 tests only for a Practical-specific generator-family advantage; it does not rule out overall stylistic/distributional alignment (as the paper concedes). To support the claim of a 'critical disconnect' between human and model reasoning, the authors should provide an independent validation subset where fresh annotators generate labels from scratch (without AI-drafted options) and compare their agreement with the gold. They should also repo
  2. [Appendix E.4 / Table 3] Human performance is reported as the majority vote of the same three annotators who helped create and validate the labels. These annotators were trained on the benchmark's own definitions and options, and they likely had prior exposure to the specific instances during the annotation/filtering phase. This creates a risk of inflated human scores (memory, confirmation bias) and makes the human-model gap not directly comparable to an independent human estimate. The paper should evaluate a separate set of naive but qualified annotators on a random subset, using the same interface, and report their agreement with the gold labels and their EM/F1. If the authors cannot collect new annotations, they should at least disclose this limitation clearly in the main text, not only in an appendix.
  3. [§4.2, Table 2, Appendix E.7] The story-level consistency metric requires all questions in a story to be answered exactly correctly, which is an extremely strict conjunction. The paper reports that the best model achieves only 0.80% full consistency, but it does not report the corresponding human story-level consistency. Without human performance on the same strict metric (or on the softer story EM/F1 given in Appendix E.7), the claim that models have a distinctive 'critical disconnect' between static recognition and dynamic sequential reasoning is not directly supported—human instance-level performance being higher does not establish that humans maintain full-story consistency substantially better. The authors should either provide human story-level metrics (even on a subsample) or soften the claim to 'models do not achieve perfect consistency over full stories' without attributing the gap specifically to a human-li
  4. [Algorithm 1, Table 14, §3.3] The construction pipeline forces exactly one practical option per theory label (8 options for Maslow, 16 for Reiss), each instantiated to the story. This introduces an unstated axiom: that for every behavior, each theoretical need/desire can be represented by one practical option, and that selecting that option is semantically equivalent to selecting the corresponding theory label in the Definition task. Yet the paper uses Definition vs. Practical as a meaningful contrast (Table 3), and the practical options are AI-generated and then only filtered for 'logical soundness' and 'alignment' (Figure 2). If this one-to-one mapping is not reliable (e.g., if a practical option conflates two needs, or if an annotator would prefer a different instantiating phrase), the cross-task comparison is confounded. I recommend validating the mapping by having fresh annotators map generated options back to t
minor comments (5)
  1. [Abstract / §1] The abstract describes MULTIVATIONBENCH as 'the first human-annotated benchmark' for this setting, but the construction is AI-drafted and human-ratified. Consider using 'human-verified' or 'human-validated' to avoid overclaiming human origination.
  2. [Table 3] The table header has a formatting issue: 'Maslow (8) Def. Mot.' appears as columns and rows; consider splitting the header for readability.
  3. [Appendix E.6 / Table 7] The same-behavior context ablation samples 150 final behavior points, which is small relative to 4,023 instances. Please provide the sampling details and confidence intervals for the reported EM/F1 differences, since the conclusion 'models do not consistently exploit distant earlier context' is based on fairly small numbers.
  4. [Appendix B.1] The contamination probe paraphrases only the textual story; visual contamination remains untested. Please state this limitation explicitly in the main contamination discussion.
  5. [§2.1] When positioning against MotiveBench, the paper states it 'typically presents isolated, single-turn scenarios.' Adding a citation to the specific evaluation setting would strengthen the claim.

Circularity Check

2 steps flagged

Practical Motivation tasks are Definition tasks under a fixed label-to-option bijection, and the human ceiling is computed from the same annotators who ratified AI-drafted gold labels.

specific steps
  1. self definitional [Appendix D.2, Table 14 (Motivation Analyst Prompt); Table 15 (Validator Prompt); §4.3]
    "Pattern: Every option must be a single sentence: "(theory_label) {target_character} desire_phrase[practical goal supported by story actions]." ... Verify Maslow Option A is Physiological, B is Safety, etc. Verify Reiss Option A is Exercise, B is Eating, etc."

    The generator is required to create exactly one practical option per theory label, and the validator enforces a fixed ordering A↔Physiological, B↔Safety, A↔Exercise, B↔Eating, etc. Therefore the Practical Motivation task is a paraphrase of the Definition task under a bijective option-alphabet map: the gold label set determines the gold practical-option set and vice versa. The §4.3 interpretation that the two formats 'probe partially distinct capabilities' is not supported—Table 3's Practical-vs-Definition gaps measure sensitivity to the paraphrase and option presentation, not an independent sequential-motivation-reasoning axis.

  2. self definitional [§3.3, Fig. 2; Appendix D.1; Appendix E.4; Appendix B.2]
    "Two state-of-the-art MLLMs, Grok-4.1 and Gemini-3, independently scanned the images and story context ... Three annotators, all graduate students from English-speaking universities, were recruited for the annotation process. ... To measure human performance on MULTIVATIONBENCH, we employ three graduate-student annotators to complete the same benchmark evaluation tasks as the models. ... We compute the final human predictions by majority vote over the three annotators’ responses."

    The gold labels are drafted by Grok-4.1/Gemini-3 and only ratified by the annotators, and the 'human ceiling' is then measured on those same annotators via majority vote. Thus the benchmark's ground truth and human reference are both downstream of one AI-seeded interpretive scheme; two evaluated model families come from the drafting families. The headline 60–79% human EM vs ≤39% model EM therefore partly conflates human-like social reasoning with agreement with an AI-drafted labeling style. The paper's own negative control (B.2) concedes it does 'not claim that this fully rules out all forms of stylistic or distributional alignment.'

full rationale

The model-evaluation numbers themselves are self-contained and reproducible: the released instances, prompts, and scoring are fixed, and the zero-shot results stand as measurements on that benchmark. The circularity is in the benchmark's validity chain for the paper's interpretation. First, the Practical Motivation tasks are not an independent task axis: the construction prompt forces exactly one practical option per theory label and the validator enforces a fixed A/B/... ordering, so Practical is Definition with paraphrased options; any divergence in Table 3 is a paraphrase effect, not a distinct 'grounding in sequential narratives' capability. Second, the human ceiling is measured on the same three annotators who established/ratified the AI-drafted gold labels, so the human-vs-model gap is not an independent external standard, and two tested models come from the drafting families. Appendix B.2 tests only for an extra Practical-specific advantage and explicitly does not rule out stylistic/distributional alignment. These issues do not make the benchmark worthless, and the contamination analysis is a reasonable check, but the central 'critical disconnect' claim rests on a standard that is partly defined by the construction pipeline. Score 6: partial by-construction reduction, not full circularity.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 1 invented entities

No numeric free parameters are fitted anywhere: this is an evaluation benchmark, not a model with trained coefficients. The only hand-chosen numeric design constants that shape a secondary finding are the story-length strata. The load-bearing assumptions are the validity of the two psychological taxonomies as label spaces, the recoverability of latent motivation from visual narratives by the annotation protocol, and the one-option-per-label generation constraint that makes Practical tasks near-equivalent to Definition tasks in label semantics. No new particles, forces, or formal objects are postulated; the only invented construct is the story-instantiated practical-option format.

free parameters (1)
  • Narrative-length strata (Short 2–5, Medium 6–9, Long 10+) = 2–5 / 6–9 / 10+
    Hand-chosen cutoffs that define the 'Impact of Narrative Length' result and the dataset statistics (§3.4). Figure 3 itself suggests the performance breakpoint is near 13 images, not 10, so a different split would shift where 'decline' is said to begin. Not fitted to model performance, but load-bearing for a secondary claim.
axioms (4)
  • domain assumption Maslow's Expanded 8-level Hierarchy of Needs is a valid operational label space for behavior→motivation mapping.
    Invoked in §3.2 and Appendix A.2; the paper cites Maslow (1970) but no empirical validation/critique literature for the taxonomy, on which the Definition tasks' correctness rests.
  • domain assumption Reiss's 16 Basic Desires is a valid operational label space for the same mapping.
    Invoked in §3.2 and Appendix A.3; the paper cites Reiss (2004) but not the psychometric literature on the Reiss Motivation Profile.
  • domain assumption Latent motivation is recoverable from visual narratives by the annotation protocol (three annotators ratifying AI-proposed chains/options).
    Core to Section 3.3 and Appendix D; there is no external, falsifiable handle on 'true' motivation, so label validity is internal to the protocol.
  • ad hoc to paper One practical option per theory label preserves semantic equivalence between Definition and Practical tasks.
    Algorithm 1 and Table 14 force exactly 8 or 16 options, each mapped one-to-one onto a theory label, so Practical tasks reduce by construction to Definition tasks with paraphrase; the paper interprets them as 'partially distinct capabilities' (§4.3).
invented entities (1)
  • Practical motivation option (story-instantiated) no independent evidence
    purpose: Context-specific instantiation of each Maslow/Reiss label in the Practical tasks; each of the 8/16 options is forced to correspond to exactly one theory label (Algorithm 1, Table 14).
    A test-design construct rather than a physical or formal entity: it has no falsifiable handle outside the paper, and its main effect is to make the Practical tasks inherit the Definition task's label structure — a design redundancy.

pith-pipeline@v1.3.0-daily-deepseek · 30911 in / 23206 out tokens · 222856 ms · 2026-08-01T15:16:21.940100+00:00 · methodology

0 comments
read the original abstract

Multimodal Large Language Models have sparked significant interest due to their potential for social intelligence; however, their ability to perform sequential motivation reasoning remains insufficiently studied. Existing evaluations predominantly examine static text or isolated visual snapshots, which do not reflect the cumulative nature of real-world behavioral drivers. To address this gap, we introduce MultivationBench, a benchmark designed to rigorously evaluate multimodal motivation reasoning within story-driven visual narratives. The benchmark builds upon established psychological frameworks - Maslow's hierarchy and Reiss's basic desires - and requires models to integrate accumulated multimodal context to infer evolving motivations. Results indicate that MultivationBench presents a significant challenge: all tested models struggle to maintain consistent motivation reasoning across sequential contexts, revealing a critical disconnect between static recognition capabilities and the dynamic reasoning essential for human-like social understanding.

Figures

Figures reproduced from arXiv: 2607.26465 by Chunkit Chan, Haochen Shi, Hao Liang, Janet Hui-wen Hsiao, Kai Chung Wong, Kawai Chung, Qing Zong, Tianshi Zheng, Weiqi Wang, Xi Yang, Yangqiu Song, Yauwai Yim, Yifan Gao, Yixuan Fu, Yuxuan Liu.

Figure 1
Figure 1. Figure 1: A Maslow motivation example from MULTIVATIONBENCH. Given the accumulated story context and images up to I11, Debbie reaching for her phone is plausibly interpreted as reflecting Cognitive needs; with additional context up to I13, which reveals family photographs and her social isolation at the dinner, the inferred motivation shifts to Love and Belonging. This example shows how the inferred practical motiva… view at source ↗
Figure 2
Figure 2. Figure 2: AI–human pipeline for constructing MULTIVATIONBENCH. We summarize the stages here and defer prompts and implementation details in Appendix D. accumulated story context and images up to that point to formulate four tasks: Maslow Definition, Maslow Practical Motivation, Reiss Definition and Reiss Practical Motivation. The Definition tasks ask models to classify the motivation of the behav￾ior based on the st… view at source ↗
Figure 3
Figure 3. Figure 3: Sequential Motivation Reasoning Performance. Performance across increasing story lengths. Left: F1 score(%). Right: Exact Match (EM) Accuracy(%) ple, o4-mini drops from 44.1% EM accuracy on Maslow Definition tasks to 24.7% on Maslow Prac￾tical Motivation task, while Llama-4-Maverick￾17B declines from 47.3% to 21.7%. This pattern suggests that these models retain substantial fa￾miliarity with psychological … view at source ↗
Figure 4
Figure 4. Figure 4: Label-wise F1 across the eight Maslow need categories (left) and the 16 Reiss desire categories (right) [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Grouped bar chart comparing the frequency [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Overview of the framework provided to the annotators. Panel (a) illustrates the interface for initial [PITH_FULL_IMAGE:figures/full_fig_p026_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 7 linked inside Pith

  1. [1]

    Logical soundness of the options

  2. [2]

    Correctness of the answer based on visual and narrative context

  3. [3]

    Based on the visual information and the story provided, which [theory label(s)] is/are most strongly expressed or fulfilled by the behavior of [character]?

    Strict alignment with Maslow’s and Reiss’s definitions to minimize hidden biases from a single model. The feedback is compiled by a Validator (Grok- 4.1-Fast-reasoning) to select the highest-quality candidates and form the final option sets. The prompt templates for the Motivation Analyst, Re- viewer, and Validator for both Maslow and Reiss tasks are pres...

  4. [4]

    answer":

    ONLY output the option you choose, No explanation or justification is required. THE ONLY output format is JSON String: {"answer": "option(s) that you choose"} Table 18: The raw prompt input for the Maslow Practical task, containing 8 context-specific options generated based on Maslow’s hierarchy. 28 Maslow Definition Visual Reasoning Prompt The following ...

  5. [5]

    11 Haoran Li, Yulin Chen, Zihao Zheng, Qi Hu, Chunkit Chan, Heshan Liu, and Yangqiu Song

    Privacy in large language models: Attacks, de- fenses and future directions.CoRR, abs/2310.10383. 11 Haoran Li, Yulin Chen, Zihao Zheng, Qi Hu, Chunkit Chan, Heshan Liu, and Yangqiu Song. 2025a. Sim- ulate and eliminate: Revoke backdoors for genera- tive large language models. InThirty-Ninth AAAI Conference on Artificial Intelligence, Thirty-Seventh Confe...

    Pith/arXiv arXiv 2025

  6. [6]

    Constrained reasoning chains for enhancing theory-of-mind in large language models. InPRI- CAI 2024: Trends in Artificial Intelligence - 21st Pacific Rim International Conference on Artificial Intelligence, PRICAI 2024, Kyoto, Japan, November 18-24, 2024, Proceedings, Part II, volume 15282 of Lecture Notes in Computer Science, pages 354–360. Springer. Zon...

    2024

  7. [7]

    Yuhang Ma, Wenting Xu, Chaoyi Zhao, Keqiang Sun, Qinfeng Jin, Xiaoda Yang, Zeng Zhao, Changjie Fan, and Zhipeng Hu

    Analyzing leakage of personally identi- fiable information in language models.CoRR, abs/2302.00539. Yuhang Ma, Wenting Xu, Chaoyi Zhao, Keqiang Sun, Qinfeng Jin, Xiaoda Yang, Zeng Zhao, Changjie Fan, and Zhipeng Hu. 2025. Storynizor: Consistent story generation via inter-frame synchronized and shuffled id injection.Proceedings of the AAAI Conference on Ar...

    Pith/arXiv arXiv 2025

  8. [8]

    InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 1011–1031, Singapore

    Towards a holistic landscape of situated theory of mind in large language models. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 1011–1031, Singapore. Association for Computational Linguistics. Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. 2024. Video-ChatGPT: Towards detailed video understanding via large vi...

    2023

  9. [10]

    InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 13691–13701

    Mlvu: Benchmarking multi-task long video understanding. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 13691–13701. Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou. 2024. Storydiffusion: Con- sistent self-attention for long-range image and video generation.CoRR, abs/2405.01434. Qing Zong,...

    Pith/arXiv arXiv 2024

  10. [14]

    Output:JSON string only

    De-duplication:For continuous behaviors across multiple frames, record the behavioronlyin the starting frame (e.g., only log “sit” in frame 1, not 2–4). Output:JSON string only. Table 10: The Extractor Prompt used in data preprocessing to identify main characters and validate behaviors. 23 Reviewer Prompt Template Role:You are an Expert Psychologist and N...

  11. [15]

    The man”) to specific names found in the text (e.g., “John

    Character Selection:Select max 3 main characters based on narrative frequency and visual significance. Prioritize the top 2 if many appear. 4.Behavior Grouping:Treat groups performing identical actions as a single character unit. 5.Conciseness:Provide brief, context-aligned behavior descriptions per frame. 6.Presence Check:Exclude characters who do not ap...

  12. [16]

    Implied Context:Refine context for transition frames where human behavior is implied by the plot but not explicitly visible. 11.Motivation Generation: Analyze the motivation of each proactive ,visible behavior made by the major character 12.Motivation Filter:Remove characters that lack motivated behaviors throughout the story. 13.Observability:Record only...

  13. [18]

    Input Analysis:Review the provided images, full story text, and the candidate behavior chains extracted by previous agents

  14. [19]

    Focus verification on these key figures

    Significance Check:Identify which characters are the most active and appear in the majority of the frames. Focus verification on these key figures

  15. [20]

    4.Validation Logic: - If a behavior or motivation is hallucinated, inaccurate, or unsupported by the story, mark the chain asInvalidand provide specific reasoning

    Grounding Verification:Rigorously check if the reported behaviors and motivations are supported by specific visual cues and the narrative context. 4.Validation Logic: - If a behavior or motivation is hallucinated, inaccurate, or unsupported by the story, mark the chain asInvalidand provide specific reasoning. - If the chain is accurate, fully grounded in ...

  16. [21]

    Fully step into the role of {target_character} and analyze their specific behavior in the context of the story

  17. [22]

    Each option must explain the practical motivation of the character corresponding to one level of the provided theory

    Create exactly [8 or 16] options. Each option must explain the practical motivation of the character corresponding to one level of the provided theory. CRITICAL GENERATION INSTRUCTIONS:

  18. [23]

    (theory_label) {target_character}desire_phrase[practical goal supported by story actions]

    Pattern:Every option must be a single sentence: “(theory_label) {target_character}desire_phrase[practical goal supported by story actions].” 2.Phrasing:Use desire phrases like “wants”, “hopes to”, “feels driven” to connect the character to the goal naturally

  19. [24]

    Based on the visual information and the story provided, what is the most likely motivation of {target_character}?

    Focus:Focus ONLY on the motivation behind the SPECIFIC behavior listed above. Do not restate the behavior itself; explain thewhy. 4.Constraint:Do NOT include the full definition text in the output options. Question Wording:“Based on the visual information and the story provided, what is the most likely motivation of {target_character}?” Output:JSON string...

  20. [25]

    Rigorously compare Set A and Set B against the criteria above

  21. [26]

    Select the superior set

  22. [27]

    OUTPUT FORMAT: Return ONLY the final, polished JSON of the selected set

    If the winner still contains minor errors (e.g., one repetitive sentence), you may apply final edits to perfect it. OUTPUT FORMAT: Return ONLY the final, polished JSON of the selected set. Table 16: The Validator Prompt used to compare two candidate option sets and select the one with higher linguistic variety and psychological accuracy. Category(Images)S...

  23. [34]

    Your output must beONLY the option letter(s) from the options provided choice :[’A’, ’B’, ’C’, ’D’, ’E’, ’F’, ’G’, ’H’]without explanation

  24. [35]

    answer":

    ONLY output the option you choose, No explanation or justification is required. THE ONLY output format is JSON String: {"answer": "option(s) that you choose"} Table 19: The raw prompt input for the Maslow Definition task, including the story context, exact definitions, and formatting constraints. 29 Reiss Practical Visual Reasoning Prompt The following is...

  25. [36]

    Based on the content provided, infer the likely motivation behind characters’ behavior, if multiple motivations are possible, select all that apply

  26. [38]

    Your output must beONLY the option letter(s) from the options provided choice :[’A’, ’B’, ..., ’P’]without explanation

  27. [39]

    answer":

    ONLY output the option you choose, No explanation or justification is required. THE ONLY output format is JSON String: {"answer": "option(s) that you choose"} Table 20: The raw prompt input for the Reiss Practical task, containing 16 options generated based on Reiss’s basic desires theory. 30 Reiss Definition Visual Reasoning Prompt The following is a Vis...

  28. [40]

    Based on the content provided, infer the likely motivation behind characters’ behavior, if multiple motivation is possible, select all that apply

  29. [41]

    Select the best answer(s) from the options

  30. [42]

    Your output must beONLY the option letter(s) from the options provided choice :[’A’, ’B’, ’C’, ’D’, ’E’, ’F’, ’G’, ’H’, ’I’, ’J’, ’K’, ’L’, ’M’, ’N’, ’O’, ’P’]without explanation

  31. [43]

    answer":

    ONLY output the option you choose, No explanation or justification is required. THE ONLY output format is JSON String: {"answer": "option(s) that you choose"} Table 21: The raw prompt input for the Reiss Definition task, asking the model to map behaviors directly to the 16 standard theoretical definitions. Mode Introduction (Prompt Header) Story Text Hand...

  32. [1970]

    is a motivational theory that explains how human needs may be prioritized and fulfilled, from basic survival to higher-order psychologi- cal growth and meaning. While Maslow origi- nally discussed five levels, later formulations com- monly expand the hierarchy; in this work we adopt an 8-level version (Physiological, Safety, Love & Belonging, Esteem, Cogn...

    2004

  33. [2012]

    InProceedings of the 2012 Joint Conference on Empirical Methods in Natural Lan- guage Processing and Computational Natural Lan- guage Learning, pages 995–1005, Jeju Island, Korea

    An empirical investigation of statistical sig- nificance in NLP. InProceedings of the 2012 Joint Conference on Empirical Methods in Natural Lan- guage Processing and Computational Natural Lan- guage Learning, pages 995–1005, Jeju Island, Korea. Association for Computational Linguistics. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke,...

    Pith/arXiv arXiv 2012

  34. [2022]

    InEuropean Conference on Computer Vision, pages 70–87

    Storydall-e: Adapting pretrained text-to-image transformers for story continuation. InEuropean Conference on Computer Vision, pages 70–87. Zainy M. Malakan, Saeed Anwar, Ghulam Mubashar Hassan, and Ajmal Mian. 2023. Sequential story- telling image dataset (SSID). Karttikeya Mangalam, Raiymbek Akshulakov, and Ji- tendra Malik. 2023. Egoschema: A diagnostic...

    Pith/arXiv arXiv 2023

  35. [2023]

    InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Sin- gapore, December 6-10, 2023, pages 14397–14413

    Fantom: A benchmark for stress-testing ma- chine theory of mind in interactions. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Sin- gapore, December 6-10, 2023, pages 14397–14413. Association for Computational Linguistics. Gnther Knoblich and Natalie Sebanz. 2008. Evolving intentions for social inter...

    Pith/arXiv arXiv 2023

  36. [2024]

    Tombench: Benchmarking theory of mind in large language models. InProceedings of the 62nd Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 15959–15983. Association for Computational Lin- guistics. Jiayang Cheng, Lin Qiu, Tsz Ho Chan, Tianqing Fang, Weiqi Wang,...

    2024

  37. [2025]

    Zheye Deng, Chunkit Chan, Weiqi Wang, Yuxi Sun, Wei Fan, Tianshi Zheng, Yauwai Yim, and Yangqiu Song

    Deepseek-r1: Incentivizing reasoning capa- bility in llms via reinforcement learning.Preprint, arXiv:2501.12948. Zheye Deng, Chunkit Chan, Weiqi Wang, Yuxi Sun, Wei Fan, Tianshi Zheng, Yauwai Yim, and Yangqiu Song. 2024. Text-tuple-table: Towards information integration in text-to-table generation via global tuple extraction.CoRR, abs/2404.14215. Zheye De...

    Pith/arXiv arXiv 2024