{"id":"69e3bd61-cf3e-4d0b-99ef-db5a96bcf449","arxiv_id":"2506.14224","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Attention heads in multimodal LLMs linearly encode agents' beliefs, and steering those heads along probe-derived directions improves first- and second-order belief accuracy on the new GridToM benchmark.","lead":"This paper introduces GridToM, a multimodal video-and-text test for theory of mind, and probes the attention heads of video-language models to show that belief information is linearly encoded inside them. The authors then steer the most informative attention heads during inference, boosting accuracy on belief questions without retraining the models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Probing final-token activations for belief correctness may decode answer color or door state rather than ToM; without control probes, neither the representational claim nor the steering improvement is established.","rationale":"The reader's weakest assumption, that GridToM's TB/FB labels are correct, is legitimate: the Appendix C.2, Figure 11, True Belief block describes a closed door and answer 'red', which is the classic false-belief outcome, while the False Belief block describes an open door and answer 'purple', the true-belief outcome. However, even if this is a typo and the dataset labels are correct, the core interpretability claim remains vulnerable to a more direct confound: the probes are trained on final-token activations, the exact position where the model's next-token prediction is formed, and the labels are tightly correlated with the answer color and with perceptual door state. Without control probes on answer color, door state, or shuffled labels, high probing accuracy and the clean clusters in Figure 5 do not establish that the model represents others' beliefs; they may only show that the answer is linearly decodable from the penultimate hidden state. Similarly, the intervention in Eq. 4 may simply push the answer distribution toward the correct color rather than modifying a ToM computation. This concern is testable with the paper's own code and data, and it directly targets the paper's strongest claim, so it should be the primary condition for accepting the interpretability result. I agree with the reader's CONDITIONAL verdict, but for a different reason than the label inversion, hence UNCHANGED and partial agreement.","tokens_in":21094,"tokens_out":7749,"duration_ms":88562,"concrete_test":"Run a control probing experiment on the same GridToM activations used in Section 4.3: train separate logistic-regression probes to predict (a) the correct answer color or option string and (b) the door open/closed state, using identical final-token activation, layer, and head selection. If either control probe reaches accuracy comparable to the belief-label probe and selects the same top-K heads, the belief probe is confounded. Then apply the Section 4.4 steering procedure with a control direction from the answer-color probe and from a shuffled-label probe at the same alpha and K values used in Table 1. If the control steering improves TB/FB/Both accuracy by more than roughly half the reported gains, the intervention is generic answer-direction shifting rather than ToM enhancement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central interpretability claim rests on linear probes trained on attention-head activations extracted at the final token of the prompt (Section 4.2). At this position the model is about to emit its answer, and final-token representations are known to encode next-token and answer information. In GridToM, the Yp/Yo labels are defined through the belief question's answer, and the correct answer color is systematically determined by whether the protagonist did or did not see the white agent's relocation. A logistic-regression probe can therefore reach high validation accuracy by decoding the answer color, the door open/closed state, or other surface correlates, without encoding any belief state. Figure 5(C)'s four separated clusters would also arise if the probe read off the correct option. The steering intervention (Eq. 4) then amplifies exactly this direction, shifting the model's answer distribution and improving accuracy on the same benchmark, which is closer to answer steering than to evidence of enhanced ToM. The MMToM-QA generalization probe (Appendix H) uses the same final-token activations and text-only labels, so it does not remove the confound. The TB/FB label inconsistency in Appendix C.2, Figure 11, is a separate benchmark-integrity issue, but even with labels fixed the probing and steering design still needs an explicit control.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents GridToM, a new multimodal theory-of-mind benchmark built on a 2D grid world, with videos and text annotations covering first- and second-order true/false belief tasks. It evaluates several MLLMs and LLMs on GridToM, reports human accuracy, then probes attention-head activations with logistic regression classifiers trained on belief-correctness labels (Yp/Yo). Selected attention heads are used in a training-free intervention (Eq. 4) that adds a scaled probe direction to attention outputs, and the paper reports accuracy gains on GridToM for LLaVA-Next-Video and Qwen2-VL. The paper also probes the MMToM-QA dataset as an external generalization check. The main claims are that MLLMs linearly encode belief states across perspectives and that steering along probe-derived directions improves ToM behavior.","tokens_in":21295,"tokens_out":6803,"duration_ms":64710,"significance":"If the claims hold, the paper would be a useful contribution: GridToM provides a controlled, perceptually grounded multimodal ToM benchmark with explicit perspective annotations, which is a strength relative to video datasets without ground-truth perception; the multi-layer attention-head probing offers a concrete interpretability method; and the inference-time intervention is lightweight and does not require fine-tuning. The inclusion of human baselines and multiple MLLMs/LLMs is also a strength. However, as written the core interpretability and intervention claims are not yet established because of the TB/FB label inconsistency, the absence of control probes, and unreported intervention hyperparameters.","major_comments":[{"comment":"There is a direct inconsistency between the dataset definitions and the example annotations. Section 3.1 and Section 5.1 define TB as the protagonist having full visibility (door open) and FB as limited visibility (door closed). However, in the 'True Belief' block of Figure 11 the yellow agent closes the green door and 'cannot see anything happening outside the green room', so the yellow agent's belief ('red') differs from the omniscient truth ('purple'); this is a false belief by the paper's own definition, not a true belief. The 'False Belief' block has the door open and the belief matching the omniscient end state. If the dataset follows Figure 11, the TB/FB labels are inverted and the Table 1 comparison (e.g., ChatGPT4O 6.2% TB vs 100% FB) would have the opposite interpretation; if Figure 11 is a typo, the ambiguity still makes the benchmark and the probe labels (which use these TB/FB distinctions) unreliable. This must be resolved by correcting the figure/annotations and by releasing a label audit with the dataset.","section":"Appendix C.2, Figure 11; Section 3.1"},{"comment":"The probing targets are not specified consistently. Section 4.2 defines both Yp and Yo and then states 'we only use the correct and incorrect labels from the protagonist's perspective', while Appendix B.1 (Figure 6) assigns four distinct label combinations (Yp,Yo) for TB and FB and describes different extraction strategies for first- and second-order beliefs. In addition, Figure 5(A) captions say 'on the test set' but Section 4.3 says the figure shows 'validation accuracy', and Section 4.4 says the top K heads are selected on the validation set. The paper needs to state exactly which labels and which data split were used to train the probes, to select heads, and to produce the reported accuracies; using test-set accuracy for head selection would make the reported probe accuracies optimistically biased.","section":"Section 4.2, Section 4.3, Appendix B.1, Figure 5(A)"},{"comment":"The confound control is missing for both the representational and the intervention claims. The probes are trained on activations extracted at the final token of the prompt, immediately before the model produces a two-choice answer, and the labels are the correctness of that answer in GridToM. Because in GridToM the correct answer color is fully determined by whether the protagonist saw the white agent's relocation, a linear probe can achieve high accuracy by decoding the answer color, the door state, or another surface feature, without encoding any belief state; the four separated clusters in Figure 5(C) would also arise if the probe read off the correct option. The intervention of Eq. (4) then amplifies exactly this direction and is evaluated only on the same GridToM benchmark, which is compatible with answer steering rather than with enhanced ToM. The MMToM-QA generalization probe in Appendix H uses the same final-token activations and text-only labels and therefore does not remove the confound. The paper should include control probes (e.g., decoding answer color or door state, probing with shuffled labels) and an evaluation of the intervention on a held-out or adversarially modified set of scenarios.","section":"Section 4.3, Eq. (4), Figure 5(C)"},{"comment":"The intervention results in Table 1 are not reproducible as reported. Eq. (4) introduces the scale α and Section 4.4 defines the number of selected heads K, but the '+α' rows in Table 1 do not state the values of K and α used, nor whether the protagonist-perspective or omniscient-perspective direction was used for each model and task. Appendix E provides parametric plots but not the exact settings that produced the headline gains (e.g., LLaVA +21.2 on first-order Both and Qwen2-VL +33.8 on first-order TB). Please report the exact K, α, and direction for every reported row, and include a sensitivity table so the reader can verify that the chosen hyperparameters are inside the stable region.","section":"Table 1, Section 5.1, Appendix E"}],"minor_comments":[{"comment":"The phrase 'adjusting in the direction of the attention head' is vague; please specify that the intervention shifts activations along the probe weight direction of the selected attention heads.","section":"Abstract, Section 5.2"},{"comment":"Section 3.1 says each video has 'approximately 40 frames', while Figures 8 and 9 show 36-frame sequences; please unify the frame counts.","section":"Section 3.1, Appendix C.1"},{"comment":"Several typos need correction: 'GirdToM' in Table 1 and elsewhere, 'interference' for 'intervention' in Section 5.1 and Appendix E, 'demonstreted' and 'alse' in Section 5.1, and 'leds' in the Introduction.","section":"Throughout"},{"comment":"In Equation (1), 'the n-th head' should be 'the h-th head', and the notation P_h^l is unclear; please define it explicitly as the projection matrix for head h at layer l.","section":"Eq. (1)"},{"comment":"The claim that α is effective roughly within [-50,50] appears inconsistent with Figure 17, whose x-axis spans [-100,100] for Qwen2-VL; please clarify whether the effective range is model-dependent.","section":"Appendix E"},{"comment":"Initial-belief accuracy in Table 3 should be accompanied by valid-response rates, since the text notes that DeepSeek-VL2-small achieved 5.9% accuracy with 89.9% invalid responses; reporting accuracy alone is misleading without this context.","section":"Section 3.2, Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The TB/FB inconsistency in Figure 11 is the single most important issue; if the figure reflects the actual dataset, the main empirical results reverse. I recommend asking the authors to provide the dataset and a complete label audit, and to rerun the baseline, probing, and intervention tables once the labels are verified. The probe-confound issue is also substantial and would require new control experiments, but it is addressable within the scope of a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"GridToM is a genuinely useful new resource, and the paper's research question is the right one. But the headline result—that attention heads linearly encode belief states and that steering them improves ToM—does not survive contact with the final-token probing setup. The probes are trained on the final token of the prompt, at the point where the model is about to produce its answer. In this dataset the correct answer color correlates with whether the protagonist saw the relocation, which correlates with the door state. A logistic regression probe could reach high accuracy by decoding the door state or answer color, without encoding any belief. The paper provides no control probes (door-state probe, answer-color probe, shuffled labels, middle-token probes). The MMToM-QA generalization uses the same final-token activations and text-only labels, so it does not clear the confound. The four clusters in Fig 5(C) could equally be explained by these surface correlates. The steering intervention then amplifies exactly that direction, so the accuracy gains in Table 1 are consistent with answer steering, not enhanced ToM. This is a serious gap, not a nitpick.\n\nSecond, Appendix C.2 Figure 11 has the TB and FB labels swapped relative to the definitions in Section 3.1 and C.1: the block labeled True Belief has a closed door and limited visibility (which should be FB), and the block labeled False Belief has an open door and full visibility (TB). The answers in the figure are also inconsistent with the labels. If the dataset follows the figure, the benchmark's central TB/FB comparison is inverted and the baseline results (e.g., 6% TB vs 100% FB for GPT-4o) would flip in meaning. Even if it is a typo, the dataset must be released with corrected labels before the results can be interpreted.\n\nThird, the intervention numbers in Table 1 omit the K and alpha values used; the appendix shows ranges but not the exact settings for the reported results. That is a reproducibility gap, though minor compared with the first two.\n\nThe paper does things right: the dataset design includes per-agent perceptual annotations; the human baseline and multi-model evaluation are useful; and probing + steering for multimodal ToM is a reasonable combination to try. The related work is appropriately covered. But the current evidence does not support the abstract's claim.\n\nI'd send this to review—the dataset and question justify referee time—but I'd expect major revision: add control probes, fix/annotate the labels, release code/data, and report hyperparameters. As it stands, it is a promising dataset with an unestablished interpretability claim.","headline":"A useful new multimodal ToM dataset, but the final-token probing setup and an appendix labeling swap leave the headline interpretability claim unestablished.","tokens_in":21865,"tokens_out":5506,"would_cite":false,"duration_ms":55136,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Attention heads in multimodal large language models encode other agents' beliefs in linearly decodable directions, and steering those directions at inference time improves Theory of Mind performance.","keywords":["theory of mind","multimodal large language models","attention head probing","linear probes","inference-time intervention","belief reasoning","grid world benchmark","interpretability"],"falsifier":"Re-derive the ground-truth belief for a sample by simulating the caption and video (the protagonist's door state and the white agent's path determine the belief); if the published example's pairing (closed door, belief in red equals 'True Belief'; open door, belief in purple equals 'False Belief') appears in the actual dataset rather than being a typo, the TB/FB labels are inverted and the reported accuracy gap and probe labels would reverse.","tokens_in":20839,"feed_emoji":"🧠","tokens_out":7423,"duration_ms":74088,"temperature":0.7,"pith_summary":"This paper sets out to show that multimodal large language models do not merely answer Theory of Mind questions by pattern matching: they build internal, linearly decodable representations of what each agent believes. To test this, the authors build GridToM, a 2-D grid-world dataset of 1,296 video-text pairs with true-belief and false-belief stories, controlled door-based perception, and first- and second-order belief questions. Probing attention-head activations with logistic regression reveals many heads whose outputs separate true from false beliefs, clustered by perspective, and the probe weight vectors act as steering directions: adding $\\alpha$ times the standard deviation along those directions during inference lifts belief-reasoning accuracy on both tested open models. The paper reads this as evidence that ToM-like reasoning is realized in attention structure and can be enhanced without fine-tuning.","feed_headline":"Steering attention heads boosts theory of mind in multimodal LLMs","feed_subtitle":"Belief states are linearly decodable inside the models, and shifting activations along those directions improves test accuracy.","key_machinery":"The load-bearing object is the attention-head activation and its probe-derived direction. For each head at each layer, the activation at the final token position is fed to a logistic-regression probe whose label is whether the protagonist's belief matches ground truth; heads whose probes exceed a threshold are the model's belief-sensitive heads, and their probe weight vectors supply the intervention direction $\\theta^h_l$. The intervention modifies the residual update as $T_{l+1} = T_l + \\sum_h (\\mathrm{Attn}^h_l(P^h_l T_l) + \\alpha\\sigma^h_l\\theta^h_l)W^o_l$, scaling the direction by $\\alpha$ standard deviations after attention and before the output projection. The GridToM dataset is the controlled setting that makes this possible: door open and closed states determine whether the protagonist sees critical events, giving clean perspective-dependent labels.","core_discovery":"The central claim is that attention heads in MLLMs carry the model's representation of who knows what: linear probes trained on head activations classify whether a belief is true from the protagonist's perspective and from an omniscient perspective, with many heads reaching high accuracy and forming four cleanly separated clusters (true/false times protagonist/omniscient). In the paper's own terms, this shows that MLLMs develop intermediate representations reflecting multi-perspective information extraction and belief inference, and that these implicit beliefs are linearly decodable. The same probe directions are then reused as interventions: after computing attention in a selected head, the model adds $\\alpha\\sigma\\theta$, where $\\theta$ is the probe weight vector and $\\sigma$ the activation standard deviation along it, which shifts reasoning toward the protagonist's perspective. On LLaVA-Next-Video-7B and Qwen2-VL-7B, this training-free steering improves first-order true-belief accuracy from 53.2% to 63.8% and 26.6% to 60.4%, respectively, with similar gains on second-order beliefs and on the joint TB-and-FB metric.","pith_inferences":["A natural stress test would swap the door state in a fixed story and check whether probe accuracy and intervention gains track the changed ground truth; if they do not, the reported effect may be a dataset artifact rather than a belief-direction mechanism.","Because the intervention is a single linear shift, it likely captures only the dominant belief feature; combining multiple probe directions or probing at intermediate tokens could reveal nested second-order belief structure that the current final-token readout misses.","The same probe-then-steer recipe could be applied to other social-cognitive dimensions, such as desires or intentions, whenever a dataset supplies perspective-controlled labels; GridToM's ATOMs-style design is a template for constructing them."],"forward_implications":["If the claim holds, ToM failures shown by closed-source models on question-answering do not prove absence of ToM; models can encode correct belief states internally and still answer wrong.","Belief representations being linearly decodable means a training-free, parameter-free intervention can improve ToM behaviour at inference time on open-weight models.","The method transfers beyond GridToM: probing on the real-world MMToM-QA dataset also finds perspective-separating heads, so the mechanism is not an artifact of grid-world rendering.","The reported TB/FB asymmetry (near-perfect FB, near-chance TB) implies benchmark format and visual-textual alignment, not just reasoning ability, determine measured ToM.","Because the intervention works by aligning attention with the protagonist's perspective, it also offers a diagnostic: heads whose probes fire on perspective information can be used to trace when a model is using the wrong agent's viewpoint."],"supporting_citations":[{"why":"Provides MMToM-QA, the real-world multimodal ToM dataset used as the external-probing validation target and as the baseline whose perceptual ambiguity motivates GridToM.","marker":"(Jin et al., 2024)"},{"why":"Supplies the inference-time intervention recipe (probing a direction then adding it scaled by activation standard deviation) that the paper adapts to attention heads.","marker":"(Li et al., 2024)"},{"why":"Defines the unexpected-transfer false-belief task that GridToM reproduces in a 2D grid world.","marker":"(Baron-Cohen et al., 1985)"},{"why":"Provides the ATOMs taxonomy of mental-state tasks that structures GridToM's first- and second-order belief questions.","marker":"(Beaudoin et al., 2020)"},{"why":"Supplies the MultiGrid library used to generate the grid-world videos and perspective-controlled agent trajectories.","marker":"(Oguntola et al., 2023)"},{"why":"Provides the underlying Minigrid environment and rendering that GridToM builds on.","marker":"(Chevalier-Boisvert et al., 2023)"},{"why":"Supplies the zero-shot QA evaluation protocol the paper follows for its baselines.","marker":"(Shapira et al., 2024)"}],"fun_headline_variants":["Attention heads encode beliefs; steering them boosts theory of mind","Probe-guided steering of attention heads improves multimodal ToM","Belief states are linearly decodable in multimodal LLMs","Training-free attention steering lifts theory of mind scores"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that GridToM's True Belief and False Belief labels are correct: the paper's own Appendix C.2 example labels a closed-door story as True Belief and an open-door story as False Belief, which would invert the protagonist's actual belief if the dataset follows that example, and every probe label and TB-versus-FB accuracy gap in the paper depends on those labels.","fun_headline_variants_meta":{"raw":{"variants":["Attention heads encode beliefs; steering them boosts theory of mind","Probe-guided steering of attention heads improves multimodal ToM","Belief states are linearly decodable in multimodal LLMs","Training-free attention steering lifts theory of mind scores"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000793,"raw_usage":{"total_tokens":3492,"prompt_tokens":940,"completion_tokens":2552,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":2486}},"tokens_in":556,"tokens_out":2552,"duration_ms":22014,"temperature":1.0,"reasoning_tokens":2486,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:18:20.291419+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-derive the ground-truth belief for a sample by simulating the caption and video (the protagonist's door state and the white agent's path determine the belief); if the published example's pairing (closed door, belief in red equals 'True Belief'; open door, belief in purple equals 'False Belief') appears in the actual dataset rather than being a typo, the TB/FB labels are inverted and the reported accuracy gap and probe labels would reverse.","supporting_citations":[{"cited_title":"theory of mind","cited_arxiv_id":null,"evidence_quote":"Defines the unexpected-transfer false-belief task that GridToM reproduces in a 2D grid world."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ATOMs taxonomy of mental-state tasks that structures GridToM's first- and second-order belief questions."},{"cited_title":"H., Zhou, X., Choi, Y., Goldberg, Y., Sap, M., and Shwartz, V","cited_arxiv_id":null,"evidence_quote":"Supplies the zero-shot QA evaluation protocol the paper follows for its baselines."}],"review_version":1}