Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

Are Language Models Sensitive to Morally Irrelevant Distractors?

T0 review · 4 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that morally irrelevant emotional context—a pleasant or unpleasant image or narrative prepended to a moral dilemma—can shift LLM moral judgments by over 30%, even in unambiguous cases.

desk verdict Situationist moral-distractor study with a useful new dataset and careful ablations, but the central causal claim is undercut by a missing matched-length control and a neutral condition that nearly reproduces the negative effect for the headline model. read the letter →

arxiv 2602.09416 v2 pith:7H7TGXCD submitted 2026-02-10 cs.CL cs.CY

Andrew Shaw , Christina Hahn , Catherine Rasgaitis , Yash Mishra , Alisa Liu , Natasha Jaques , Yulia Tsvetkov , Amy X. Zhang This is my paper · ORCID
classification cs.CLcs.CY
keywords moraldistractorssituationismLLMjudgmentcontextualalignmentbenchmarksemotionalvalenceMMAPAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper challenges the common assumption that LLMs hold stable moral preferences by testing whether morally irrelevant emotional context can change their judgments. The authors curate 60 'moral distractors'—text and images with positive, neutral, or negative emotional valence—and prepend them to two existing moral benchmarks. Across several LLMs, negative distractors consistently reduce the probability of choosing a prosocial action and increase harsh verdicts, sometimes by more than 30% even in low-ambiguity scenarios. If correct, this undermines the reliability of existing moral benchmarks and argues for context-aware AI alignment rather than treating models as fixed moral reasoners.

What carries the argument

The central object is the 'moral distractor'—an emotionally valenced piece of text or image that has no moral bearing on the scenario—plus the Marginal Moral Action Probability (MMAP), defined as the probability of selecting the rule-following action divided by the summed probabilities of both forced-choice actions. The MMAP quantifies how much the distractor shifts action selection relative to a no-distractor baseline, and is the metric on which the headline 30% effect is measured.

What would settle it

Run the same moral scenarios with a matched-length, emotionally flat narrative (e.g., a list of furniture descriptions) as the control condition. If negative distractors no longer cause a significantly larger drop in moral action probability than the flat narrative, the claim that emotional valence specifically drives the effect is falsified.

Watch

Extended reading notes

Core claim

The central discovery is that LLMs' moral judgments are strongly sensitive to morally irrelevant emotional context. Prepending a positive, neutral, or negative sentence or image before a moral scenario shifts the model's choice of action or verdict in a valence-dependent way: negative distractors lower the marginal probability of selecting the rule-following action and increase 'everyone sucks here' verdicts, while positive distractors often have the opposite effect. The shifts reach over 30 percentage points in unambiguous scenarios for some models, mirroring the 'situationist' finding in human moral psychology that incidental factors like ambient noise or pleasant smells affect moral behav

Load-bearing premise

The paper's causal attribution to emotional valence assumes that prepending a distractor changes only the emotional character of the prompt; the control is a no-distractor prompt, not a matched-length neutral narrative, so part of the observed shift could be a generic response to extra context rather than specifically moral valence.

Editorial extensions

If this is right

  • If LLM moral judgments are this context-sensitive, single-snapshot moral benchmark scores cannot be interpreted as stable value measurements; a model's apparent ethics could depend on incidental prompt color.
  • Deploying LLMs in emotionally negative settings—mental-health support, content moderation, conflict mediation—could systematically bias their choices toward less prosocial actions, so context and prompt design become safety-relevant.
  • Enabling reasoning modes substantially mitigates the effect in low-ambiguity scenarios, suggesting inference-time reasoning is a concrete partial remedy.
  • Safety and alignment evaluations should stress-test guardrails under varied distractors, particularly negative ones, since alignment constraints may fail under incidental emotional context.
  • Moral responsibility shifts toward the developers and deployers who shape the contexts in which models operate, rather than treating the model itself as the moral agent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because neutral textual distractors also produced large drops in some models' moral action probability, a plausible extension is to test matched-length, emotionally flat narratives as controls; this would isolate whether the effect is specifically due to emotional valence or partly a generic response to added context.
  • Beyond the paper: the situationist analogy suggests a testable prediction that repeated or accumulated distractors in a multi-turn conversation would compound the shift; no multi-turn setup is tested here.
  • Beyond the paper: the authors only tested visual distractors on one model, so a cross-modal, cross-model comparison would reveal whether the bias is modality-general or an artifact of the particular image set.
  • Beyond the paper: an actionable extension is to explicitly instruct the model that preceding context is irrelevant to the moral question; if that restores stable judgments, it would provide a cheap guardrail for sensitive deployments.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper asks whether LLMs, like humans in situationist moral psychology, are sensitive to morally irrelevant emotional context. The authors curate 60 textual and visual 'moral distractors' from IDEST and OASIS, categorize them by valence (positive, neutral, negative), and prepend them to two moral benchmarks: MORALCHOICE (forced-choice actions) and r/AITA (verdict classification). Across four model families, they report that negative distractors lower the marginal moral action probability (MMAP) in low-ambiguity MORALCHOICE scenarios, in one case by over 30%, and increase the share of 'everyone sucks here' (ESH) verdicts in r/AITA. They conclude that LLMs show situationist moral biases and that alignment must be context-aware. The paper provides a new dataset and experimental protocol, but the central causal claim—that the valence, rather than the mere presence and format of appended context, drives the shifts—is undermined by the lack of a matched-length/format neutral control.

Significance. If the valence-specific effect is real, this is a valuable contribution connecting situationist moral psychology to LLM evaluation, with clear implications for alignment, deployment in sensitive settings, and the interpretation of moral benchmarks. The paper also provides a reusable distractor curation pipeline and extensive ablations over model size, instruction tuning, and reasoning, which are strengths. However, the headline claim—that morally irrelevant emotional context shifts LLM moral judgments—requires evidence that the shifts are due to the valence of the distractor rather than to generic prompt perturbation. The current control condition does not permit that attribution, so the significance of the findings is conditional on a follow-up matched control experiment.

major comments (4)
  1. [§3.2, Table 5, A.1.1] The control condition is a no-distractor prompt, not a matched-length neutral narrative. For LLAMA-3.2-3B-INSTRUCT in low-ambiguity scenarios, MMAP drops from 96.20% at baseline to 75.71% with neutral textual distractors, already a ~21% relative reduction, and to 66.51% with negative distractors (Table 5). Thus neutral distractors reproduce most of the negative-distractor drop. The distractor prompts are long, second-person, sensory narratives (see A.1.1) while the baseline is a short, direct question; this confounds valence with length, narrative voice, formatting, and domain shift. The r/AITA condition similarly inserts a first-person scene into the system prompt (A.1.2), changing the instruction context. A matched-length, matched-format neutral control that varies only valence is needed to support the claim that the shifts are caused by moral valence rather than by added context.
  2. [§4.1] The statistical reporting is incomplete. The text states p<0.05 for certain differences but does not provide the actual p-values, effect sizes, or confidence intervals for the MMAP differences. More importantly, no test is reported for the neutral vs. negative contrast, which is essential to establish that negative distractors have a valence-specific effect beyond the generic effect of adding a narrative. Without such tests, the claim that negative distractors are uniquely harmful is under-supported.
  3. [§4.2, Table 8] The interpretation that 'negative distractors consistently cause models to become more disapproving of others' behavior' is not fully supported by the verdict distributions. For LLAMA-3.2-3B-INSTRUCT, negative distractors decrease YTA from 72.4% to 36.9% and increase NTA from 20.0% to 46.1%, i.e., the model becomes more supportive of the original poster, even though the ESH share increases. The consistent effect is specifically on ESH, not on overall disapproval. This should be reframed to avoid overstating the directional pattern.
  4. [§3.1] The 'moral irrelevance' of the distractors is not validated against the specific benchmark scenarios. The filtering criteria (e.g., excluding extreme emotional content) are applied at the distractor level, but a distractor such as a 'foul smell' could plausibly be interpreted as part of the scenario environment in some MORALCHOICE items, potentially affecting the model's situational reasoning. A human validation study confirming that each distractor is morally irrelevant to each benchmark scenario would strengthen construct validity.
minor comments (4)
  1. [Throughout] There are several typos: 'judgeents' in §3.1, 'Naseline' in Table 8, 'Itâ C™s' in A.1.2, and 'youself' in A.1.1. The paper would benefit from proofreading.
  2. [Figures 2–3] The figures show point estimates without error bars or confidence intervals. Adding them would aid the reader in assessing the stability of the MMAP differences.
  3. [Table 7] The footnote states 'Smaller dataset of 50 scenarios used' for the reasoning ablation, but it is not clear which scenarios or why the subset was chosen. Please clarify.
  4. [§3.2] The text says 'we generate one baseline no-distractor response and one response with each distractor' for r/AITA, but given 30 distractors (10 per valence), the exact number of sampled responses per scenario should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the MMAP and verdict measurements are direct experimental comparisons, not derived from the quantities they claim to predict.

full rationale

The paper contains no derivation chain that reduces to its inputs. The central metric, MMAP, is defined directly from token logits as p(af)/(p(af)+p(av)) (Eq. 4) and is compared across baseline, positive, neutral, and negative distractor conditions; no parameter is fitted to the outcome and then reported as a prediction. The distractor dataset is curated from external human-annotated valence norms (IDEST and OASIS), not from the models' moral judgments, so the valence labels are not derived from the moral-correctness labels used to score responses. The r/AITA verdict shifts are likewise direct classifier outputs over LLM responses. Self-citations (e.g., Gonen et al. 2025 on semantic leakage, Köprü et al. 2025 on expression leakage) are contextual and not load-bearing for the paper's measurements. The neutral-distractor confound visible in Table 5 — where neutral textual distractors lower Llama-3.2-3B-Instruct's low-ambiguity MMAP from 96.20% to 75.71%, nearly matching the negative condition's 66.51% — is a threat to the causal attribution to valence, but it is a control/validity limitation, not a circularity: the reported MMAP values are empirical facts regardless of interpretation. No step meets the required standard of exhibiting a specific reduction of a claimed result to its own inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the operationalization of 'moral distractor'—treating emotional valence as an independent variable via hand-picked cutoffs and prompt prepending—and on accepting prior benchmark annotations as ground truth. No fitted parameters or new entities are introduced beyond the dataset and the two hand-chosen selection rules.

free parameters (2)
  • Textual distractor valence cutoffs = negative 1-4, neutral 4-6, positive 6-9 on IDEST 1-9 scale
    Hand-chosen bins define the independent variable; no robustness check across cutoffs, and 'neutral 4-6' overlaps the boundaries.
  • Visual distractor selection rule = top 10 by valence, bottom 10, and 10 closest to median (OASIS)
    Hand-picked sampling rule; no analysis of how this selection affects results.
assumptions (4)
  • domain assumption The curated IDEST/OASIS items are morally irrelevant to the benchmark scenarios.
    Section 3.1 filters by valence and excludes extreme/moral narratives, but no human validation confirms irrelevance in the context of the moral dilemmas; a foul-smell narrative could shift inferred situation, not just mood.
  • domain assumption Prepending a distractor to a prompt is a valid operationalization of incidental situational context for an LLM.
    Section 3.2 uses text/image prefixing as the analog of human ambient conditions; there is no check that this mimics the human situationist experiments.
  • domain assumption MORALCHOICE and r/AITA annotations are accepted as ground truth for moral action and judgment.
    Section 3.2 relies on prior benchmark labels (rule-following/violating, AITA verdicts) without re-auditing.
  • domain assumption Token-probability aggregation over answer orderings measures moral preference.
    Section 3.2 averages selection probabilities over two orderings; assumes these are stable and independent of prompt-sensitive decoding.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Are Language Models Sensitive to Morally Irrelevant Distractors?." pith.science (2026). https://pith.science/paper/7H7TGXCD

@misc{pith2026260209416,
  author       = {Pith},
  title        = {Pith review of: Are Language Models Sensitive to Morally Irrelevant Distractors?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7H7TGXCD}},
  note         = {Machine review of arXiv:2602.09416}
}
read the original abstract

With the rapid uptake of large language models (LLMs) across high-stakes settings, it is becoming increasingly important to ensure that LLMs behave in ways that align with human values. Existing moral benchmarks for this purpose often prompt LLMs with value statements, moral scenarios, or psychological questionnaires, with the implicit underlying assumption that LLMs report somewhat stable moral preferences. However, moral psychology research has shown that even human moral judgements are sensitive to morally irrelevant situational factors such as the smell of cinnamon rolls or the level of ambient noise, thereby challenging moral theories which assume that human moral judgements are stable. Here we draw inspiration from this "situationist" view of moral psychology to evaluate whether LLMs exhibit similar cognitive moral biases. We curate a novel multimodal dataset of 60 "moral distractors" from existing psychological datasets of emotionally-valenced images and narratives, which have no moral relevance to the situation presented. After injecting these distractors into existing moral benchmarks, we find that moral distractors can shift the moral judgements of LLMs by over 30% even in unambiguous scenarios, highlighting the instability of LLMs' moral judgements and the need for more contextual approaches to AI alignment.

Figures

Figures reproduced from arXiv: 2602.09416 by the authors.

Figure 1
Figure 1. Example of an immoral response induced by [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. In low-ambiguity scenarios with textual distractors, negative distractors decrease the marginal probability of a moral action (MMAP) compared to the baseline no distractor condition by up to 30+%, while positive distractors have less of an effect due to prior model alignment. baseline (no distractor) condition in forced-choice scenarios for each moral rule, and then aggregate and average across all moral rules ( [P… view at source ↗
Figure 4
Figure 4. Example of immoral response induced by neg [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Across high- and low-ambiguity scenarios, [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: In response to everyday moral dilemmas from r/AITA with textual distractors, negative distractors consistently increase the share of “everyone sucks here” (ESH) verdicts and induce the largest shifts in verdict distribution. distribution deviates from baseline for that…
Figure 7
Figure 7. Figure 7: Marginal moral action probability (MMAP) [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Marginal moral action probability (MMAP) [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: In low-ambiguity ablation experiments over model size, no consistent relationship is observed between model size and sensitivity to textual distractors [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: In low-ambiguity ablation experiments over instruction-tuning, no consistent relationship is observed between instruction-tuning and sensitivity to textual distractors. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: In low-ambiguity ablation experiments over reasoning, reasoning mitigates but does not elimi￾nate the effect of textual distractors on marginal moral action probability (MMAP). 20 [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: In response to everyday moral dilemmas from /AITA, textual distractors do not have a statistically [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Scalable Approach to Evaluating Moral Sensitivity in LLMs

    cs.CY 2026-07 conditional novelty 6.5 of 10

    Under morally irrelevant noise, eight LLMs preserve the semantic content of identified moral features above calibrated floors, despite significant changes in feature counts.

  2. Normative Robustness as a Frontier for Non-Verifiable Reasoning in LLMs

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Frontier LLMs exhibit moral deliberative sycophancy by shifting their moral reasoning and justifications up to 6.5% on average toward a user's stated preferred view in simulated deliberations.

  3. Are LLMs Bad at Moral Reasoning?

    cs.CY 2026-06 unverdicted novelty 5.0 of 10

    Reanalyzing MoReBench by assigning LLMs the task of generating scoring rubrics shows better calibration to human rubrics and suggests stronger LLM moral reasoning than previously reported.

Reference graph

Works this paper leans on

7 extracted references · 4 linked inside Pith · cited by 3 Pith papers

  1. [4]

    InProceedings of Recent Advances in Natural Language Process- ing, page 578–586, Varna, Bulgaria

    Am i blue or is my hobby counting teardrops? expression leakage in large language models as a symptom of irrelevancy disruption. InProceedings of Recent Advances in Natural Language Process- ing, page 578–586, Varna, Bulgaria. Association for Computational Linguistics (Shumen, Bulgaria). Yuxuan Li, Hirokazu Shirado, and Sauvik Das. 2025. Actions speak lou...

    arXiv 2025

  2. [6]

    Lorenzo Zangari, Candida M

    Qwen3 technical report.arXiv preprint arXiv:2505.09388. Lorenzo Zangari, Candida M. Greco, Davide Picca, and Andrea Tagarelli. 2025. ME2-BERT: Are events and emotions what you need for moral foundation predic- tion? InProceedings of the 31st International Con- ference on Computational Linguistics, pages 9516–

    arXiv 2025

  3. [2021]

    In International Conference on Learning Representa- tions

    Aligning {ai} with shared human values. In International Conference on Learning Representa- tions. Jianchao Ji, Yutong Chen, Mingyu Jin, Wujiang Xu, Wenyue Hua, and Yongfeng Zhang. 2025. Moral- bench: Moral evaluation of llms.SIGKDD Explor. Newsl., 27(1):62–71. Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny T. Liang, Sydney Levine, ...

    2025

  4. [2022]

    InThe Oxford Handbook of Moral Psychology

    Situationism, moral improvement, and moral responsibility. InThe Oxford Handbook of Moral Psychology. Oxford University Press. 10 An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others

  5. [2024]

    InProceedings of the 2024 Conference on Empiri- cal Methods in Natural Language Processing, pages 17737–17752, Miami, Florida, USA

    Moral foundations of large language models. InProceedings of the 2024 Conference on Empiri- cal Methods in Natural Language Processing, pages 17737–17752, Miami, Florida, USA. Association for Computational Linguistics. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altma...

    arXiv 2024

  6. [2025]

    Yu Ying Chiu, Michael S

    A comprehensive review of llm-based content moderation: Advancements, challenges, and future directions.Knowledge-Based Systems, page 114689. Yu Ying Chiu, Michael S. Lee, Rachel Calcott, Bran- don Handoko, Paul de Font-Reaulx, Paula Rodriguez, Chen Bo Calvin Zhang, Ziwen Han, Udari Mad- hushani Sehwag, Yash Maurya, Christina Q Knight, Harry R. Lloyd, Flo...

    arXiv 2025

  7. [9532]

    Am I the Asshole

    Association for Computational Linguistics. 11 A Experimental Details A.1 Example Prompts A.1.1 MORALCHOICEPrompt system You are an AI assistant that responds to every question in a particular format. Answer rules: - Your answer vocabulary is lim- ited to A or B. - Your answer is not allowed to start with ’As an AI language model ...’ or with ’I cannot ......

    2004

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.