Pith. sign in

REVIEW 3 major objections 6 minor 3 cited by

Mechanistic Interpretability of Emotion Inference in Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Pith's one-line read This paper argues that LLM emotion inference is functionally localized to mid-layer attention units and can be causally steered by modulating appraisal vectors, aligning outputs with psychological appraisal theory.

desk verdict The localization story is well-built and worth taking seriously; the appraisal-steering evidence is circular on the current data and needs an independent holdout design. read the letter →

arxiv 2502.05489 v2 pith:DMUFEOZR submitted 2025-02-08 cs.CL cs.AI

classification cs.CLcs.AI
keywords mechanisticinterpretabilityemotioninferenceappraisaltheoryactivationpatchinglinearrepresentationhypothesiscausalsteeringmulti-headself-attentionlargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that large language models do not spread emotion processing evenly across the network: emotion inference is functionally localized to specific middle layers, and within those layers the attention units do the decisive work. The authors argue that the internal computations correspond to psychological appraisal dimensions—evaluations such as pleasantness, self-agency, and predictability—and that these dimensions are linearly encoded, so adding or subtracting their latent directions shifts the model's output emotion in theoretically expected ways. The payoff is causal, not just descriptive: if the claim holds, emotional text generation can be steered without retraining, and emotion inference in LLMs becomes inspectable through the lens of established emotion theory.

What carries the argument

The load-bearing object is the residual-stream hidden state $h_t^{(l)}$ at layer $l$ and token $t$, together with its additive components from multi-head self-attention, $a_t^{(l)}$, and the feed-forward block, $m_t^{(l)}$. Linear probes defined by $W^\top x + b$ measure where emotion is decodable, while causal activation patching replaces $a_t^{(l)}$ or $m_t^{(l)}$ at a chosen layer to see whether the source emotion transfers. For appraisal steering, each appraisal is assigned a regression vector $v_a$; the unique effect of appraisal $a$ is isolated as $z_a = (I - P_{-a})v_a$ by projecting out all other appraisal vectors, and the intervention adds $\beta z_a / \|z_a\|_2$ to the hidden state, with positive or negative $\beta$ promoting or demoting the concept. This machinery converts an interpretability claim into a causal claim: a concept is said to be present where probing finds it and functionally responsible where patching and steering change the decision.

What would settle it

Take a model from the paper's list and run the same activation-patching recipe on a dataset where the prompt and task are identical but the emotion labels are replaced by shuffled labels: if mid-layer patching still transfers the shuffled labels at the same rates, the 'emotion units' are carrying syntactic or format information rather than emotion. Alternatively, compare steering with the appraisal vector $z_a$ against steering with a random vector of the same norm; if a random vector at layer 9 produces the same distribution shifts, the appraisal direction is not the cause.

Watch

Extended reading notes

Core claim

Across ten autoregressive models spanning four families and sizes from one billion to fourteen billion parameters, the paper reports that a linear probe trained on the last token's hidden state reaches near-peak accuracy by the mid-layers, and activation patching transfers an emotion label from one vignette to another only when applied to the multi-head self-attention outputs in roughly those same layers. Attention visualization shows the last token's heads start attending to emotionally loaded words at those layers, and a control task with identical syntax locates its critical units elsewhere, so the mid-layer focus is not a generic artifact. For Llama 3.2 1B, the paper further reports that appraisal concepts are linearly decodable from hidden states around the same region, that cosine similarity between emotion and appraisal vectors follows appraisal-theory mappings, and that adding an orthogonalized appraisal direction at layer 9 shifts the output distribution in line with those mappings—promoting pleasantness raises joy and pride, demoting other-agency raises guilt, and so on. The paper takes these results to show that LLM emotion inference is a localized, appraisal-like computation that is steerable at inference time.

Load-bearing premise

The whole steering argument assumes emotions and appraisals are stored as linear directions in the model's internal vectors, so the probe directions name the true concepts and moving along them changes those concepts; if the internal code is nonlinear, the probes could be labeling superficial correlates of human ratings instead.

Editorial extensions

If this is right

  • Emotion labels can be transferred between examples by patching only a few mid-layer attention units, meaning the model's emotion decision is concentrated enough to target surgically.
  • Adding or subtracting appraisal directions at the critical layer changes the predicted emotion in theory-consistent ways, so emotional output can be steered without fine-tuning or prompt engineering.
  • The localization pattern repeats across ten models of different families and sizes, suggesting the mid-layer attention mechanism is a general architectural feature rather than a quirk of one checkpoint.
  • Promoting two appraisals together shifts the emotion distribution to the conjunction predicted by appraisal theory, giving a finer control knob than single-dimension steering.
  • The same units are not critical for a syntactically identical task, indicating the localization is tied to emotion semantics rather than prompt structure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the linear-representation premise fails for emotion features, the probe directions used here could be linear correlates of human labels rather than the model's actual internal code; testing with dictionary-learned or non-linear features would settle which story is right.
  • The same appraisal-vector machinery could generalize to other subjective constructs LLMs are asked to output, such as personality traits, moods, or moral judgments, wherever an analogous theory supplies dimensions to probe.
  • The observed orthogonality of emotion and appraisal vectors in final layers suggests that later layers are doing something other than appraisal reasoning; if confirmed, this constrains where to look for the final decision in future interpretability studies.
  • One testable extension: feed the steered generations to human raters without the closed-set label menu; if the appraisal shifts also change perceived emotion in free text, the steering is not confined to forced-choice classification.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper applies mechanistic interpretability techniques to study how autoregressive LLMs infer emotions from short narrative vignettes in the crowd-enVENT dataset. Across ten models from six families, the authors train linear probes on MHSA, FFN, and residual-stream activations, perform activation patching, knockout interventions, and attention analyses, and conclude that emotion-relevant processing is functionally localized to mid-layer MHSA units. They then train linear appraisal regressors on the same dataset, compute cosine similarities between appraisal and emotion directions, and steer generation by adding residual stream vectors, reporting that the resulting emotion shifts align with appraisal-theoretic expectations. The paper includes prompt-robustness checks, an isomorphic control task, and a random-vector steering control, and makes code available. The authors explicitly acknowledge in Section 11 that the linear representation hypothesis and the causal direction of appraisal-emotion relations remain open.

Significance. If the localization claim holds, this is a useful contribution: it extends mechanistic interpretability beyond synthetic template tasks to naturalistic emotional text, provides cross-family and cross-scale evidence for mid-layer emotion processing, and the triangulation via probing, patching, knockout, attention visualization, and an isomorphic control is a genuine methodological strength. The steering results, if valid, would also offer a practical avenue for controllable affective generation. However, the appraisal-steering evidence is currently weakened by a train/evaluation overlap with the same annotation source, and the central steering claim is conditional on a linearity assumption that the paper flags but does not test for appraisal concepts. The localization claim itself is more robust and is the part most likely to survive scrutiny; the psychological-plausibility claim needs a stronger, non-circular validation.

major comments (3)
  1. [§7, §9, Figure 5] The appraisal-steering evidence is potentially circular: the appraisal vectors v_a are trained as linear regressions on crowd-enVENT appraisal ratings (Section 7), the expected appraisal-emotion associations used to judge the steering outcome are computed by averaging appraisal scores per emotion label in the same dataset (Section 7, Figure 5), and the steering experiment is evaluated on examples from that same corpus (Section 9). If the probes merely capture the empirical appraisal-emotion correlations present in the training annotations, then adding the probe direction to a hidden state will push outputs toward the emotions that share those dataset correlations, independent of whether the model internally represents appraisals as causally efficacious concepts. To break this circularity, I recommend evaluating the steering on a held-out or independent dataset, or constructing expected shifts from one half of the data while fitting probes on the other half, and comparing against a null model that uses the same procedure with shuffled appraisal labels.
  2. [§3, Appendix A.3 and C.6] The localization, probing, and steering analyses are performed only on examples where the LLM's prediction matches the human self-reported emotion label (Section 3, Appendix A.3), which selects for unambiguous or easily classified items and may inflate the apparent mid-layer consolidation. The authors acknowledge this bias (Appendix A.3, C.6) and provide a preliminary comparison of appraisal patterns, but they do not show how the central localization results (Figures 2 and 3) would change on the misclassified subset or on a random sample of all examples. Since the paper's abstract makes a general claim about emotion representations in LLMs, this selection bias is load-bearing; I would like to see either a direct replication on a broader sample or a quantitative argument that the filter does not alter the layer-wise conclusions.
  3. [§9, §11, Appendix E] The steering experiment assumes that appraisal concepts are linearly encoded as residual-stream directions, but this assumption is not validated for the appraisal probes: Appendix E reports R² values for linear regression, which only show linear detectability, not that the recovered direction is the model's internal causal feature, especially given recent findings that not all features are linear (Engels et al., 2025, cited in Section 11). The authors acknowledge this limitation in Section 11, but the acknowledgement does not include a concrete test of whether a nonlinear probe or an alternative causal intervention would produce the same emotion shifts. I recommend adding a control where appraisal modulation is performed with directions from a nonlinear probe (e.g., a kernelized or MLP-based probe) or with directions fit on a different task, to demonstrate that the steering effect is not an artifact of the linear approximation.
minor comments (6)
  1. [Section 6, Appendix C.2] The patching window size is fixed at five layers across all models, despite large differences in model depth; while Appendix C.2 explores spans 1, 3, and 5 for Llama 3.2 1B, a similar justification for the fixed span used in Figure 3 would strengthen comparability across models.
  2. [Section 7, Figure 5] The appraisal-emotion associations in Figure 5 are presented as point averages without confidence intervals, and some emotion classes have only 250 samples in the dataset (Appendix A.1); adding bootstrap intervals or sample sizes would help the reader judge the stability of these associations.
  3. [Section 8] The cosine similarity analysis between appraisal and emotion vectors is only shown for Llama 3.2 1B; reporting the same analysis for at least one other model family would support the claim that the observed appraisal-emotion decoupling is not idiosyncratic.
  4. [Section 9, Appendix F] The steering results in Figure 7 and Appendix F report distribution shifts over all examples but do not include statistical significance tests or confidence intervals; adding them, for instance via bootstrap resampling, would make the comparisons across β values more convincing.
  5. [Appendix B, Section 7] There are minor typographical errors, such as 'MSHA' instead of 'MHSA' in Appendix B and 'anunpleasant' in Section 7; a careful proofreading pass is needed.
  6. [Section 11] The limitation paragraph on nonlinear features could be more directly connected to the appraisal-steering pipeline: it is the appraisal vectors, not just the emotion probes, that are assumed to be linear, and a sentence making that explicit would help readers calibrate the strength of the steering conclusions.

Circularity Check

1 steps flagged · score 6.0 of 10

Appraisal-steering 'predictions' are scored against dataset-derived associations that also trained the appraisal probes.

  1. fitted input called prediction [Section 7 ('Investigating Appraisal Concepts') and Section 9 ('Intervention on Appraisal Concepts'), Figures 5 and 7]
    "These mappings, which align closely with prior findings in appraisal theory (Wondra and Ellsworth, 2015), are extracted from our dataset by averaging appraisal scores for each emotion label. [...] Notably, we observe a remarkable alignment with theoretical and intuitive expectations. For instance, increasing the pleasantness appraisal promotes both joy and pride, aligning with the fact that both of these emotions have high associations with the pleasantness appraisal."

    The appraisal probe vectors va are trained by regression to predict crowd-enVENT appraisal ratings from model hidden states, and the emotion classifier W is trained on the same dataset's emotion labels. The 'theoretical and intuitive expectations' used to score steering success are not an external benchmark: Figure 5 is explicitly computed from the same dataset by averaging the same appraisal ratings for each emotion label. The steering direction za is a function of va, and Section 8 defines sim(a,e) = va^T we, so the predicted emotion shift (e.g., pleasantness promotes joy/pride) is essentially the sign of the fitted inner product between the appraisal probe and the emotion probe, both fit on the same labels.

full rationale

The functional-localization claims (probing, activation patching, knockout, attention patterns) are self-contained: they use held-out probe accuracy, causal interventions, and prompt-robustness controls, and they do not reduce to their inputs. The circularity is confined to the appraisal-steering/psychological-plausibility claim. The appraisal probe vectors are fitted to crowd-enVENT appraisal ratings, and the expected emotion shifts are computed by averaging the same appraisal ratings per emotion label in the same dataset. The steering experiment then shows that adding the fitted probe direction shifts the model's emotion distribution in the direction of those dataset-derived associations. This is a consistency check of the fitted probes, not an independent confirmation from appraisal theory. The paper's Limitations acknowledge the linear-representation assumption and the unresolved causal direction between appraisals and emotions, but they do not flag the train/evaluation overlap between probe fitting and the expectations used to validate steering. Because the central localization result remains independent, the overall circularity is partial rather than total.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim rests on fitted probe directions: the emotion probe weight matrix W and the appraisal regression vectors v_a are trained on hidden states from the same filtered dataset and then treated as concept vectors. The steering uses a residualized combination of those fitted vectors. The paper also assumes the linear representation hypothesis, the reliability of self-reported emotion labels as ground truth, and that dataset-averaged appraisal-emotion associations are a valid proxy for appraisal theory. These are not independent external benchmarks.

free parameters (4)
  • emotion probe weight matrix W = N/A (learned weights, no closed form)
    Trained by linear classification on hidden states to predict emotion labels; the columns w_e are treated as emotion concept vectors in Sections 8 and Appendix D, so the steering results depend on this fit.
  • appraisal probe vectors v_a = N/A (learned weights, no closed form)
    Trained by linear regression to predict each human appraisal rating from hidden states; these directions define the appraisal concepts and are used for cosine similarity and for the steering intervention in Section 9.
  • steering scaling factor beta = 1, 2, 4
    Chosen by hand to set the strength of appraisal modulation; the observed distribution shifts are reported for these values in Figure 7 and Appendix F.
  • patching and knockout span size = 3 and 5 layers
    Hand-chosen intervention window; patching success depends on span size (Figure 17), so this choice matters for the localization claims.
assumptions (5)
  • domain assumption Emotion and appraisal concepts are linearly encoded in hidden-state activations (linear representation hypothesis).
    Invoked in Sections 5 and 7 to justify linear probes and in Section 9 to justify additive steering; acknowledged as uncertain in Section 11, citing Engels et al. (2025).
  • domain assumption Self-reported emotion labels in crowd-enVENT are treated as ground truth, and only model-human agreement cases are analyzed.
    Section 3 and Appendix A.3; the paper acknowledges in Appendix C.6 that this may bias toward unambiguous emotional content.
  • domain assumption Appraisal-emotion associations computed by averaging crowd-enVENT appraisal ratings per emotion are a valid proxy for appraisal theory.
    Used to evaluate the steering results in Section 9; the associations come from the same dataset used to fit the appraisal probes, rather than an independent theoretical benchmark.
  • domain assumption The last token's activations are the relevant locus for next-token emotion prediction.
    Assumed in Section 4 and validated partially in Appendix C.3, which finds consistent signal increase from earlier to later tokens.
  • standard math Standard linear algebra of projection and residualization.
    The residualized direction z_a = (I - P_{-a}) v_a in Section 9 assumes the projection matrix is computed on the fitted probe vectors; this is pure mathematics.
invented entities (1)
  • latent appraisal concept directions z_a
    purpose: To causally steer the model's emotion by adding or subtracting a residualized appraisal direction from hidden states (Section 9).
    Constructed from probe vectors fitted to the same dataset and not validated against independent behavioral or neural data; the random-vector control in Appendix F is in-paper, not outside evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mechanistic Interpretability of Emotion Inference in Large Language Models." pith.science (2026). https://pith.science/paper/DMUFEOZR

@misc{pith2026250205489,
  author       = {Pith},
  title        = {Pith review of: Mechanistic Interpretability of Emotion Inference in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DMUFEOZR}},
  note         = {Machine review of arXiv:2502.05489}
}
read the original abstract

Large language models (LLMs) show promising capabilities in predicting human emotions from text. However, the mechanisms through which these models process emotional stimuli remain largely unexplored. Our study addresses this gap by investigating how autoregressive LLMs infer emotions, showing that emotion representations are functionally localized to specific regions in the model. Our evaluation includes diverse model families and sizes and is supported by robustness checks. We then show that the identified representations are psychologically plausible by drawing on cognitive appraisal theory, a well-established psychological framework positing that emotions emerge from evaluations (appraisals) of environmental stimuli. By causally intervening on construed appraisal concepts, we steer the generation and show that the outputs align with theoretical and intuitive expectations. This work highlights a novel way to causally intervene and precisely shape emotional text generation, potentially benefiting safety and alignment in sensitive affective domains.

Figures

Figures reproduced from arXiv: 2502.05489 by the authors.

Figure 1
Figure 1. Emotion inference through latent appraisal [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Layer-wise accuracies of emotion probe experiments across different models (each row) with varying [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results of activation patching experiments where we measure the success rate of transferring the output [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (31 more)
Figure 4
Figure 4. Figure 4: (Top) Primary prompt used in this study. Different segments of the prompt are colored differently. (Bottom) Most attended tokens at each layer from the perspective of the last token in Llama 3.2 1B. Layer 9 is the first layer in which the model attends to the tokens in…
Figure 6
Figure 6. Figure 6: Cosine similarity of emotion vectors with [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Results of appraisal concept modulation by intervening at layer 9 hidden states of Llama 3.2 1B for [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Llama 3.2 1B open vocab generation of emo [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Confusion matrix comparing the true labels to [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: The distribution of next word emotion label predictions. The self-reported human labels are shown on [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Comparison of models’ accuracy on the emo [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 14
Figure 14. Figure 14: Zero-Intervention accuracy with span 1 on the last token index of Llama 3.2 1B across all layers. There is a clear drop in accuracy when MHSA activa￾tions in layer 10 are knocked out. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Layers 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy after…
Figure 15
Figure 15. Figure 15: Random-Intervention accuracy with span 1 on the last token index of Llama 3.2 1B across all layers. A noticeable drop in accuracy is observed when MHSA activations in layer 10 are knocked out. gests that the signal is not linearly identifiable. In fact, [PITH_FULL_IM…
Figure 13
Figure 13. Figure 13: Probing test accuracy on last token of llama 3.2 1B for all layers. Top linear probe, bottom non-linear probe. There is a noticeable increase in probe performance in early layers when using a simple non￾linear probe. C.1 Details on Probing and Causal-Intervention Here…
Figure 16
Figure 16. Figure 16: Activation patching results for Llama 3.2 1B across different layers (FFN, MHSA, and hidden state units) [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]
Figure 17
Figure 17. Figure 17: Effect of span size on Llama 3.2 1B activation patching at MHSA across different layers, evaluated over [PITH_FULL_IMAGE:figures/full_fig_p021_17.png]
Figure 18
Figure 18. Figure 18: Probing test accuracy on different tokens of Llama 3.2 1B across all layers. We observe a consistent [PITH_FULL_IMAGE:figures/full_fig_p022_18.png]
Figure 19
Figure 19. Figure 19: Zero intervention accuracy on different token indices of llama 3.2 1B for all layers with span = [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]
Figure 20
Figure 20. Figure 20: Random intervention accuracy on different token indices of Llama 3.2 1B across all layers with span = [PITH_FULL_IMAGE:figures/full_fig_p023_20.png]
Figure 21
Figure 21. Figure 21: Results of activation patching success on different tokens of Llama 3.2 1B across all layers with span = 3. [PITH_FULL_IMAGE:figures/full_fig_p023_21.png]
Figure 22
Figure 22. Figure 22: Probing test accuracy of the control isomor [PITH_FULL_IMAGE:figures/full_fig_p023_22.png]
Figure 23
Figure 23. Figure 23: Control isomorphic experiment results for Llama 3.2 1B activation patching across different layers at [PITH_FULL_IMAGE:figures/full_fig_p024_23.png]
Figure 24
Figure 24. Figure 24: The distribution of next word emotion label predictions for different prompt templates and varied [PITH_FULL_IMAGE:figures/full_fig_p024_24.png]
Figure 25
Figure 25. Figure 25: Accuracy of different prompts on the emo [PITH_FULL_IMAGE:figures/full_fig_p025_25.png]
Figure 26
Figure 26. Figure 26: Probing accuracy with different prompts measured at last token in Llama 3.2 1B for all layers. This [PITH_FULL_IMAGE:figures/full_fig_p026_26.png]
Figure 27
Figure 27. Figure 27: Success of activation patching with different prompts, measured at the last token index in Llama 3.2 1B [PITH_FULL_IMAGE:figures/full_fig_p026_27.png]
Figure 28
Figure 28. Figure 28: The heatmap showing the success of direct emotion promotion when applied at different layers of [PITH_FULL_IMAGE:figures/full_fig_p027_28.png]
Figure 29
Figure 29. Figure 29: Probing results for Llama 3.2 1B, conducted separately for each appraisal dimension across different [PITH_FULL_IMAGE:figures/full_fig_p027_29.png]
Figure 30
Figure 30. Figure 30: Effect of promoting and demoting pleasantness at different layers of Llama 3.2 1B with three levels of scaling factor β. β = 0 represents the original distribution without appraisal modulation. A consistent increase in distribution shift is observed as β increases acr…
Figure 31
Figure 31. Figure 31: Effect of promoting and demoting other-agency at different layers of Llama 3.2 1B using three levels of scaling factor β. β = 0 represents the original distribution without appraisal modulation. Mid-layer appraisal modulation exhibits a theoretically plausible shift i…
Figure 32
Figure 32. Figure 32: Effect of promoting and demoting predictability at different layers of Llama 3.2 1B using three levels of scaling factor β. β = 0 represents the original distribution without appraisal modulation. Mid-layer appraisal modulation exhibits a theoretically plausible shift…
Figure 33
Figure 33. Figure 33: Superposition of pleasantness and other-agency appraisal modulation at different layers of Llama 3.2 1B. Results show successful promotion of pride with no further occurrences of joy in layers 9–11 when demoting other-agency, and the promotion of guilt and fear with n…
Figure 34
Figure 34. Figure 34: Superposition of pleasantness and predictability appraisal modulation at different layers of Llama 3.2 1B. Results show a successful transition from pride to surprise and a greater distribution of fear compared to anger in mid-layers when promoting unpredictability. j…
Figure 35
Figure 35. Figure 35: Sankey plot for pleasantness appraisal modulation when we perform it at layer 9 of Llama 3.2 1B model [PITH_FULL_IMAGE:figures/full_fig_p030_35.png]
Figure 36
Figure 36. Figure 36: Results of the control experiment in which we randomly sample a vector and add it to the hidden state of [PITH_FULL_IMAGE:figures/full_fig_p031_36.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Semantic Primes as Explanans for Emotion in Large Language Models

    cs.AI 2026-07 conditional novelty 6.0 of 10

    NSM semantic primes are more recoverable, more causally effective, and behaviorally more interchangeable with emotions than appraisal dimensions in four instruction-tuned LLMs.

  2. Fine-Grained Interpretation of Political Opinions in Large Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Four-dimensional political concept vectors learned from LLM internals can detect and partially steer political leanings better than a single left-right axis.

  3. Reconsidering LLM Uncertainty Estimation Methods in the Wild

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Most LLM uncertainty estimates degrade under distribution shift and adversarial prompts, but simple ensembling of scores at test time improves reliability.

Reference graph

Works this paper leans on

93 extracted references · 64 canonical work pages · cited by 3 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219

  4. [4]

    Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In International Conference on Learning Representations

  5. [5]

    Badr AlKhamissi, Greta Tuckute, Antoine Bosselut, and Martin Schrimpf. 2025. The LLM language network: A neuroscientific approach for identifying causally task-relevant units. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p...

  6. [6]

    Khai Loong Aw, Syrielle Montariol, Badr AlKhamissi, Martin Schrimpf, and Antoine Bosselut. 2024. https://openreview.net/forum?id=nXNN0x4wbl Instruction-tuning aligns LLM s to the human brain . In First Conference on Language Modeling

  7. [7]

    Lisa Feldman Barrett. 2017. The theory of constructed emotion: an active inference account of interoception and categorization. Social cognitive and affective neuroscience, 12(1):1--23

  8. [8]

    Yonatan Belinkov. 2018. On internal language representations in deep learning: An analysis of machine translation and speech recognition. Ph.D. thesis, Massachusetts Institute of Technology

Show all 93 references
  1. [9]

    Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207--219

  2. [10]

    Joshua Bensemann, Alex Peng, Diana Benavides-Prado, Yang Chen, Neset Tan, Paul Michael Corballis, Patricia Riddle, and Michael Witbrock. 2022. Eye gaze and self-attention: How humans and transformers attend words in sentences. In Proceedings of the Workshop on Cognitive Modeli...

  3. [11]

    Leonard Bereska and Stratis Gavves. 2024. Mechanistic interpretability for AI safety - a review. Transactions on Machine Learning Research

  4. [12]

    Joost Broekens, Bernhard Hilpert, Suzan Verberne, Kim Baraka, Patrick Gebhard, and Aske Plaat. 2023. Fine-grained affective processing capabilities emerging from large language models. In 2023 11th International Conference on Affective Computing and Intelligent Interaction (AC...

  5. [13]

    Tobias Brosch and David Sander. 2013. Comment: the appraising brain: towards a neuro-cognitive model of appraisal processes in emotion. Emotion Review, 5(2):163--168

  6. [14]

    Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Benjamin Bucknall, Andreas Haupt, Kevin Wei, J \'e r \'e my Scheurer, Marius Hobbhahn, et al. 2024. Black-box access is insufficient for rigorous ai audits. In The 2024 ACM Conference on Fairness,...

  7. [15]

    Angelica Chen, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew L Leavitt, and Naomi Saphra. 2024. Sudden drops in the loss: Syntax acquisition, phase transitions, and simplicity bias in MLM s. In The Twelfth International Conference on Learning Representations

  8. [16]

    Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does BERT look at? an analysis of BERT `s attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276--286, Florence, Italy. Ass...

  9. [17]

    Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adri \`a Garriga-Alonso. 2023. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36:16318--16352

  10. [18]

    Alexis Conneau, German Kruszewski, Guillaume Lample, Lo \"i c Barrault, and Marco Baroni. 2018. What you can cram into a single \ & ! \# * vector: Probing sentence embeddings for linguistic properties. In Proceedings of the 56th Annual Meeting of the Association for Computatio...

  11. [19]

    Oliver Eberle, Stephanie Brandl, Jonas Pilot, and Anders S gaard. 2022. Do transformer models show similar attention patterns to task-specific human gaze? In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages ...

  12. [20]

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. Toy models of sup...

  13. [21]

    Phoebe C Ellsworth and Klaus R Scherer. 2003. Appraisal processes in emotion. Oxford University Press

  14. [22]

    Zohar Elyoseph, Dorit Hadar-Shoval, Kfir Asraf, and Maya Lvovsky. 2023. Chatgpt outperforms humans in emotional awareness evaluations. Frontiers in Psychology, 14:1199058

  15. [23]

    Joshua Engels, Eric J Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark. 2025. Not all language model features are linear. In The Thirteenth International Conference on Learning Representations

  16. [24]

    Allyson Ettinger, Ahmed Elgohary, and Philip Resnik. 2016. Probing for semantic evidence of composition by means of simple classification tasks. In Proceedings of the 1st workshop on evaluating vector-space representations for nlp, pages 134--139

  17. [25]

    Jiahai Feng, Stuart Russell, and Jacob Steinhardt. 2025. Monitoring latent world states in language models with propositional probes. In The Thirteenth International Conference on Learning Representations

  18. [26]

    Nico H Frijda, Peter Kuipers, and Elisabeth Ter Schure. 1989. Relations among emotion, appraisal, and emotional action readiness. Journal of personality and social psychology, 57(2):212

  19. [27]

    Chris Gagne and Peter Dayan. 2023. The inner sentiments of a thought. arXiv preprint arXiv:2307.01784

  20. [28]

    Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. 2022. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 30--45, Abu Dhabi, Un...

  21. [29]

    Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. 2024. Patchscopes: A unifying framework for inspecting hidden representations of language models. In Forty-first International Conference on Machine Learning

  22. [30]

    Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. 2018. Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information. In Proceedings of the 2018 EMNLP Workshop B lackbox NLP : Analyz...

  23. [31]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, et al. 2024. The llama 3 herd of models

  24. [32]

    Abhijeet Gupta, Gemma Boleda, Marco Baroni, and Sebastian Pad \'o . 2015. Distributional vectors encode referential attributes. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 12--21, Lisbon, Portugal. Association for Computatio...

  25. [33]

    Michael Hanna, Ollie Liu, and Alexandre Variengien. 2023. How does GPT -2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. In Thirty-seventh Conference on Neural Information Processing Systems

  26. [34]

    Stefan Heimersheim and Neel Nanda. 2024. How to use and interpret activation patching. arXiv preprint arXiv:2404.15255

  27. [35]

    John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 27...

  28. [36]

    Dieuwke Hupkes and Willem Zuidema. 2018. Visualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18 , pag...

  29. [37]

    Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu. 2023. Evaluating and inducing personality in pre-trained language models. In NeurIPS

  30. [38]

    Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. 2024 a . P ersona LLM : Investigating the ability of large language models to express personality traits. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3605--3627,...

  31. [39]

    Yibo Jiang, Bryon Aragam, and Victor Veitch. 2024 b . Uncovering meanings of embeddings via partial orthogonality. Advances in Neural Information Processing Systems, 36

  32. [40]

    Akos K \'a d \'a r, Grzegorz Chrupa a, and Afra Alishahi. 2017. Representation of linguistic form and function in recurrent neural networks. Computational Linguistics, 43(4):761--780

  33. [41]

    Arne K \"o hn. 2015. What`s in an embedding? analyzing word embeddings through multilingual evaluation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2067--2073, Lisbon, Portugal. Association for Computational Linguistics

  34. [42]

    Philip A Kragel, David Sander, and Kevin S LaBar. 2024. Can brain data be used to arbitrate among emotion theories? In Emotion theory: The Routledge comprehensive guide, pages 511--542. Routledge

  35. [43]

    Sreejan Kumar, Theodore R Sumers, Takateru Yamakoshi, Ariel Goldstein, Uri Hasson, Kenneth A Norman, Thomas L Griffiths, Robert D Hawkins, and Samuel A Nastase. 2024. Shared functional specialization in transformer-based language models and the human brain. Nature communicatio...

  36. [44]

    Elisa Kwon, John D Patterson, Roger E Beaty, and Kosa Goucher-Lambert. 2024. Assessing the alignment between word representations in the brain and large language models. In International Conference on-Design Computing and Cognition, pages 207--223. Springer

  37. [45]

    Jinqi Lai, Wensheng Gan, Jiayang Wu, Zhenlian Qi, and S Yu Philip. 2024. Large language models in law: A survey. AI Open

  38. [46]

    Richard S Lazarus. 1991. Emotion and adaptation. Oxford University Press on Demand

  39. [47]

    Joana Leit \ a o, Ben Meuleman, Dimitri Van De Ville, and Patrik Vuilleumier. 2020. Computational imaging during video game playing shows dynamic synchronization of cortical and subcortical networks of emotions. PLoS biology, 18(11):e3000900

  40. [48]

    Omer Levy and Yoav Goldberg. 2014. Linguistic regularities in sparse and explicit word representations. In Proceedings of the eighteenth conference on computational natural language learning, pages 171--180

  41. [49]

    Li, Maxwell Nye, and Jacob Andreas

    Belinda Z. Li, Maxwell Nye, and Jacob Andreas. 2021. Implicit representations of meaning in neural language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Proces...

  42. [50]

    Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. 2023. Inference-time intervention: Eliciting truthful answers from a language model. In Thirty-seventh Conference on Neural Information Processing Systems

  43. [51]

    Diab, and Maarten Sap

    Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona T. Diab, and Maarten Sap. 2024. BIG 5- CHAT : Shaping LLM personalities through training on human-grounded data

  44. [52]

    Tom Lieberum, Matthew Rahtz, J \'a nos Kram \'a r, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. 2023. Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla. arXiv preprint arXiv:2307.09458

  45. [53]

    Xin Liu, Ziyue Zhang, and Jingxin Nie. 2025. https://arxiv.org/abs/2502.18725 Talking to the brain: Using large language models as proxies to model brain semantic representation . Preprint, arXiv:2502.18725

  46. [54]

    Ri c ards Marcinkevi c s, Sonia Laguna, Moritz Vandenhirtz, and Julia E Vogt. 2024. Beyond concept bottleneck models: How to make black boxes intervenable?

  47. [55]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35:17359--17372

  48. [56]

    Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. 2024. Circuit component reuse across tasks in transformer language models. In The Twelfth International Conference on Learning Representations

  49. [57]

    Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013 a . Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168

  50. [58]

    Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013 b . Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies , pages...

  51. [59]

    MistralAI. 2024. https://mistral.ai/news/mistral-nemo/ Mistral nemo . Accessed: 2024-12-07

  52. [60]

    Agnes Moors. 2013. On the causal role of appraisal in emotion. Emotion Review, 5(2):132--140

  53. [61]

    Neel Nanda. 2022. https://github.com/neelnanda-io/TransformerLens Transformerlens

  54. [62]

    Neel Nanda, Andrew Lee, and Martin Wattenberg. 2023. Emergent linear representations in world models of self-supervised sequence models. In Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, Singapore. Association for Computational...

  55. [63]

    Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, et al. 2024. 2 olmo 2 furious. arXiv preprint arXiv:2501.00656

  56. [64]

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. 2022. In-context learning and induction heads. arXiv preprint arXiv:2209.11895

  57. [65]

    Andrew Ortony, Gerald L Clore, and Allan Collins. 2022. The cognitive structure of emotions. Cambridge university press

  58. [66]

    Kiho Park, Yo Joong Choe, and Victor Veitch. 2024. The linear representation hypothesis and the geometry of large language models. In International conference on machine learning, ICML'24. JMLR.org

  59. [67]

    Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih

    Matthew E. Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. 2018. Dissecting contextual word embeddings: Architecture and representation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1499--1509, Brussels, Belgium. Ass...

  60. [68]

    Nikolay B Petrov, Gregory Serapio-Garc \' a, and Jason Rentfrow. 2024. Limited ability of llms to simulate human psychological behaviours: a psychometric analysis. arXiv preprint arXiv:2405.07248

  61. [69]

    Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. 2024. A practical review of mechanistic interpretability for transformer-based language models. arXiv preprint arXiv:2407.02646

  62. [70]

    I Rosenman and C Smith. 2001. Appraisal theory: Overview, assumptions, varieties, controversies. Appraisal processes in emotion. Theory, methods, research, pages 3--19

  63. [71]

    Klaus R Scherer et al. 1984. On the nature and function of emotion: A component process approach. Approaches to emotion, 2293(317):31

  64. [72]

    Ashish Sharma, Kevin Rushton, Inna Lin, David Wadden, Khendra Lucas, Adam Miner, Theresa Nguyen, and Tim Althoff. 2023. Cognitive reframing of negative thoughts through human-language model interaction. In Proceedings of the 61st Annual Meeting of the Association for Computati...

  65. [73]

    Xing Shi, Inkit Padhi, and Kevin Knight. 2016. Does string-based neural MT learn source syntax? In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1526--1534, Austin, Texas. Association for Computational Linguistics

  66. [74]

    Beau Sievers and Mark A Thornton. 2024. Deep social neuroscience: the promise and peril of using artificial neural networks to study the social brain. Social Cognitive and Affective Neuroscience, 19(1):nsae014

  67. [75]

    Craig A Smith and Phoebe C Ellsworth. 1985. Patterns of cognitive appraisal in emotion. Journal of personality and social psychology, 48(4):813

  68. [76]

    Craig A Smith and Leslie D Kirby. 2011. The role of appraisal and emotion in coping and adaptation. The handbook of stress science: Biology, psychology, and health, pages 195--208

  69. [77]

    Joseph Suh, Suhong Moon, Minwoo Kang, and David Chan. 2024. Rediscovering the latent dimensions of personality with large language models as trait descriptors. In NeurIPS 2024 Workshop on Behavioral Machine Learning

  70. [78]

    Tak and Jonathan Gratch

    Ala N. Tak and Jonathan Gratch. 2023. Is GPT a Computational Model of Emotion? . In 2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII), pages 1--8, Los Alamitos, CA, USA. IEEE Computer Society

  71. [79]

    Tak and Jonathan Gratch

    Ala N. Tak and Jonathan Gratch. 2024. Gpt-4 emulates average-human emotional cognition from a third-person perspective

  72. [80]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, et al. 2024. Gemma 2: Improving open language models at a practical size

  73. [81]

    Daniel Freeman, Theodore R

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua ...

  74. [82]

    Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019 a . BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593--4601, Florence, Italy. Association for Computational Linguistics

  75. [83]

    Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick. 2019 b . What do you learn from context? probing for sentence structure in contextualized word representations. In Intern...

  76. [84]

    Eric Todd, Millicent Li, Arnab Sen Sharma, Aaron Mueller, Byron C Wallace, and David Bau. 2024. Function vectors in large language models. In The Twelfth International Conference on Learning Representations

  77. [85]

    Enrica Troiano, Laura Oberl \"a nder, and Roman Klinger. 2023. Dimensional modeling of emotions in text with appraisal theories: Corpus creation, annotation reliability, and prediction. Computational Linguistics, 49(1):1--72

  78. [86]

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. 2023. Activation addition: Steering language models without optimization. arXiv e-prints, pages arXiv--2308

  79. [87]

    Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. Advances in neural information processing systems, 33:12388--12401

  80. [88]

    Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023 a . Interpretability in the wild: a circuit for indirect object identification in GPT -2 small. In The Eleventh International Conference on Learning Representations

  81. [89]

    Xuena Wang, Xueting Li, Zi Yin, Yue Wu, and Jia Liu. 2023 b . Emotional intelligence of large language models. Journal of Pacific Rim Psychology, 17:18344909231213958

  82. [90]

    Joshua Wondra and Phoebe Ellsworth. 2015. An appraisal theory of empathy and other vicarious emotional experiences. Psychological review, 122

  83. [91]

    interpretability illusion

    Zhengxuan Wu, Atticus Geiger, Jing Huang, Aryaman Arora, Thomas Icard, Christopher Potts, and Noah D. Goodman. 2024. A reply to makelov et al. (2023)'s "interpretability illusion" arguments

  84. [92]

    Nutchanon Yongsatianchot, Parisa Ghanad Torshizi, and Stacy Marsella. 2023. Investigating large language models’ perception of emotion using appraisal theory. In 2023 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), ...

  85. [93]

    Weixiang Zhao, Yanyan Zhao, Xin Lu, Shilong Wang, Yanpeng Tong, and Bing Qin. 2023. Is chatgpt equipped with emotional dialogue capabilities? arXiv preprint arXiv:2304.09582

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.