pith. sign in

arxiv: 2209.06687 · v3 · submitted 2022-09-14 · 💻 cs.CL

How people talk about each other: Modeling Generalized Intergroup Bias and Emotion

Venkata S Govindarajan , Katherine Atwell , Barea Sinno , Malihe Alikhani , David I. Beaver , Junyi Jessy Li This is my paper

Pith reviewed 2026-05-24 11:30 UTC · model grok-4.3

classification 💻 cs.CL
keywords interpersonal biasgroup relationshipsemotion detectionNLP biassocial mediacongressional tweetsbias detectionemotion analysis
0
0 comments X

The pith

Neural models outperform humans at identifying interpersonal group relationships in speech by using fine-grained emotion labels as supervision.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper broadens bias research beyond specific demographic targets to a generalized interpersonal group relationship (IGR) between speaker and target in an utterance. It releases the first dataset of US Congress tweets annotated for interpersonal emotions, which act as found supervision for IGR labels. Analyses establish that subtle emotional signals reliably mark different forms of bias. Neural models achieve substantially higher accuracy than humans on IGR identification, and a shared encoding between IGR and emotion tasks produces gains on both.

Core claim

By anchoring prediction of speaker-target relationships (IGR) on interpersonal emotion annotations from congressional tweets, neural models can detect generalized intergroup bias at rates far above human performance, with joint training on both tasks improving results across the board.

What carries the argument

Interpersonal emotion annotations serving as found supervision for IGR labels, combined with a shared neural encoding that transfers signals between the two tasks.

If this is right

  • Models can identify subtle biases in text without explicit mentions of demographic groups.
  • Emotional signals serve as indicators for multiple distinct forms of intergroup bias.
  • Humans perform above chance on IGR identification but are outperformed by neural models.
  • Joint training on emotion and IGR yields measurable accuracy gains for both tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method might apply to detecting relational bias in non-political online text if emotional patterns hold across domains.
  • If emotion-IGR links prove consistent, the approach could lower the cost of creating bias datasets by reusing emotion annotations.
  • Extending the annotation scheme to other languages would test whether the emotional signals for bias generalize culturally.

Load-bearing premise

Interpersonal emotion annotations provide valid found supervision for IGR labels because emotional signals are reliably indicative of different biases.

What would settle it

A new set of utterances with independently verified IGR labels where emotion-based models show no improvement over random guessing or separate training.

Figures

Figures reproduced from arXiv: 2209.06687 by Barea Sinno, David I. Beaver, Junyi Jessy Li, Katherine Atwell, Malihe Alikhani, Venkata S Govindarajan.

Figure 1
Figure 1. Figure 1: Emotions ordered by the number of examples [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Co-occurence of emotions in our dataset. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Distribution of interpersonal emotions in unsupervised representations of tweets in our dataset. Orange [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
read the original abstract

Current studies of bias in NLP rely mainly on identifying (unwanted or negative) bias towards a specific demographic group. While this has led to progress recognizing and mitigating negative bias, and having a clear notion of the targeted group is necessary, it is not always practical. In this work we extrapolate to a broader notion of bias, rooted in social science and psychology literature. We move towards predicting interpersonal group relationship (IGR) - modeling the relationship between the speaker and the target in an utterance - using fine-grained interpersonal emotions as an anchor. We build and release a dataset of English tweets by US Congress members annotated for interpersonal emotion -- the first of its kind, and 'found supervision' for IGR labels; our analyses show that subtle emotional signals are indicative of different biases. While humans can perform better than chance at identifying IGR given an utterance, we show that neural models perform much better; furthermore, a shared encoding between IGR and interpersonal perceived emotion enabled performance gains in both tasks. Data and code for this paper are available at https://github.com/venkatasg/interpersonal-bias

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces a dataset of US Congress tweets annotated for interpersonal emotions as 'found supervision' for interpersonal group relationship (IGR) labels. It analyzes how emotional signals distinguish bias types, reports that humans identify IGR above chance while neural models perform substantially better, and shows that multi-task learning with shared encoding between IGR and emotion prediction improves performance on both tasks.

Significance. If the emotion-to-IGR proxy mapping holds, the work provides a novel, broader framework for modeling generalized intergroup bias in NLP grounded in social psychology, moving beyond narrow demographic targeting. The public release of the dataset and code supports reproducibility and follow-on research in computational social science.

major comments (2)
  1. [Dataset construction] Dataset construction section: IGR labels are derived exclusively from the interpersonal emotion annotations with no reported independent validation, direct IGR annotations, or inter-annotator agreement between the derived labels and human IGR judgments. This is load-bearing for the central claims because the reported human-model performance gap and multi-task gains (abstract) become uninterpretable if the proxy is noisy or confounded.
  2. [Analyses] Analyses section: The reported differences in emotion distributions across IGR categories do not address or control for potential confounds (e.g., topic, speaker identity, or utterance length) that could drive the observed associations, weakening the claim that 'subtle emotional signals are indicative of different biases.'
minor comments (2)
  1. [Introduction] The term 'found supervision' is used in the abstract and introduction but would benefit from a clearer operational definition and discussion of its limitations relative to direct supervision.
  2. Annotation details (number of annotators, agreement metrics for the emotion labels themselves, and exact mapping rules from emotions to IGR) are referenced but not fully specified in the provided text, hindering exact replication.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address each major comment below, indicating planned revisions where appropriate.

read point-by-point responses

  1. Referee: [Dataset construction] Dataset construction section: IGR labels are derived exclusively from the interpersonal emotion annotations with no reported independent validation, direct IGR annotations, or inter-annotator agreement between the derived labels and human IGR judgments. This is load-bearing for the central claims because the reported human-model performance gap and multi-task gains (abstract) become uninterpretable if the proxy is noisy or confounded.

    Authors: The IGR labels are intentionally derived from the emotion annotations as 'found supervision,' following established mappings in social psychology literature linking specific interpersonal emotions to intergroup relationship types (e.g., positive emotions to alliance, negative to derogation). This proxy approach is core to the paper's contribution of using emotions as an anchor for broader bias modeling. No separate direct IGR annotations were collected, as the design relies on the emotion data. The human and model performance figures are evaluated against these derived labels, and the multi-task results show that joint modeling improves capture of the emotion-IGR relationships. We will revise the dataset section to explicitly detail the emotion-to-IGR mapping rules and add a limitations paragraph acknowledging the lack of independent IGR validation as a direction for future work. revision: partial

  2. Referee: [Analyses] Analyses section: The reported differences in emotion distributions across IGR categories do not address or control for potential confounds (e.g., topic, speaker identity, or utterance length) that could drive the observed associations, weakening the claim that 'subtle emotional signals are indicative of different biases.'

    Authors: We agree that controlling for confounds would strengthen the interpretability of the emotion distribution differences. In the revised manuscript, we will add supplementary analyses that control for utterance length (via regression or stratification) and speaker identity (via fixed effects where feasible given the congressional data). Topic control is more challenging without additional annotation but will be discussed as a potential confound. These additions will better isolate the emotional signals while preserving the original observed associations. revision: yes

Circularity Check

0 steps flagged

No circularity; empirical claims rest on independent annotations and standard model evaluation

full rationale

The paper is an empirical NLP study that annotates tweets for interpersonal emotions and uses those annotations as 'found supervision' to derive IGR labels. It then trains and evaluates neural models on held-out data for IGR identification and multi-task learning with emotion prediction. No mathematical derivations, equations, or parameter-fitting steps are described that reduce any reported prediction or result to its inputs by construction. The performance comparisons (models vs. humans, multi-task gains) are based on standard supervised learning with released data, not on any self-definitional mapping or self-citation chain. This matches the default case of a self-contained empirical paper with no load-bearing circular steps.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Central claim rests on annotation reliability for emotions and validity of found supervision for IGR; standard NLP modeling assumptions apply with no new entities or fitted constants named in abstract.

axioms (2)
  • domain assumption Human annotations for interpersonal emotions are reliable and capture bias signals
    Used to derive IGR labels and support analyses of emotional signals
  • domain assumption Subtle emotional signals in text are indicative of different intergroup biases
    Basis for the found supervision and model performance claims

pith-pipeline@v0.9.0 · 5744 in / 1234 out tokens · 57771 ms · 2026-05-24T11:30:14.712919+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 33 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

    work page

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

    work page

  3. [3]

    Muhammad Abdul-Mageed and Lyle Ungar. 2017. https://doi.org/10.18653/v1/P17-1067 E mo N et: F ine- G rained E motion D etection with G ated R ecurrent N eural N etworks . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 718--728, Vancouver, Canada. Association for Computational Linguistics

    work page doi:10.18653/v1/p17-1067 2017

  4. [4]

    Roee Aharoni and Yoav Goldberg. 2020. https://doi.org/10.18653/v1/2020.acl-main.692 Unsupervised Domain Clusters in Pretrained Language Models . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 7747--7763, Online. Association for Computational Linguistics

    work page doi:10.18653/v1/2020.acl-main.692 2020

  5. [5]

    Luigi Anolli, Valentino Zurloni, and Giuseppe Riva. 2006. https://doi.org/10.3200/GENP.133.3.237-255 Linguistic Intergroup Bias in Political Communication . The Journal of General Psychology, 133:237 -- 255

    work page doi:10.3200/genp.133.3.237-255 2006

  6. [6]

    Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, and Leonardo Neves. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.148 T weet E val: Unified Benchmark and Comparative Evaluation for Tweet Classification . In Findings of the Association for Computational Linguistics: EMNLP 2020 , pages 1644--1650, Online. Association for Computational...

    work page doi:10.18653/v1/2020.findings-emnlp.148 2020

  7. [7]

    David Beaver and Jason Stanley. 2018. https://doi.org/10.5840/gfpj201839224 Toward a Non-Ideal Philosophy of Language . Graduate Faculty Philosophy Journal, 39(2):503--547

    work page doi:10.5840/gfpj201839224 2018

  8. [8]

    Taylor Berg-Kirkpatrick, David Burkett, and Dan Klein. 2012. https://aclanthology.org/D12-1091 A n E mpirical I nvestigation of S tatistical S ignificance in NLP . In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning , pages 995--1005, Jeju Island, Korea. Association fo...

    work page 2012

  9. [9]

    Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020. https://doi.org/10.18653/v1/2020.acl-main.372 G o E motions: A Dataset of Fine-Grained Emotions . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 4040--4054, Online. Association for Computational Linguistics

    work page doi:10.18653/v1/2020.acl-main.372 2020

  10. [10]

    Shrey Desai, Cornelia Caragea, and Junyi Jessy Li. 2020. https://doi.org/10.18653/v1/2020.acl-main.471 Detecting Perceived Emotions in Hurricane Disasters . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 5290--5305, Online. Association for Computational Linguistics

    work page doi:10.18653/v1/2020.acl-main.471 2020

  11. [11]

    Bradley W. Gorham. 2006. https://doi.org/10.1111/j.1460-2466.2006.00020.x News Media's Relationship With Stereotyping: The Linguistic Intergroup Bias in Response to Crime News . Journal of Communication, 56(2):289--308. Place: United Kingdom Publisher: Blackwell Publishing

    work page doi:10.1111/j.1460-2466.2006.00020.x 2006

  12. [12]

    Hippel, Denise Sekaquaptewa, and P

    W. Hippel, Denise Sekaquaptewa, and P. Vargas. 1997. https://doi.org/10.1006/jesp.1997.1332 The Linguistic Intergroup Bias As an Implicit Indicator of Prejudice . Journal of Experimental Social Psychology, 33:490--509

    work page doi:10.1006/jesp.1997.1332 1997

  13. [13]

    Masahiro Kaneko and Danushka Bollegala. 2019. https://doi.org/10.18653/v1/P19-1160 Gender-preserving Debiasing for Pre-trained Word Embeddings . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 1641--1650, Florence, Italy. Association for Computational Linguistics

    work page doi:10.18653/v1/p19-1160 2019

  14. [14]

    Anne Maass. 1999. https://doi.org/10.1016/S0065-2601(08)60272-5 Linguistic Intergroup Bias: Stereotype Perpetuation Through Language . In Mark P. Zanna, editor, Advances in Experimental Social Psychology , volume 31, pages 79--121. Academic Press

    work page doi:10.1016/s0065-2601(08)60272-5 1999

  15. [15]

    Anne Maass, Daniel Anthony Salvi, Luciano Arcuri, and Gün R. Semin. 1989. https://doi.org/10.1037/0022-3514.57.6.981 Language use in intergroup contexts: the linguistic intergroup bias. Journal of Personality and Social Psychology, 57 6:981--93

    work page doi:10.1037/0022-3514.57.6.981 1989

  16. [16]

    Saif Mohammad. 2012. https://aclanthology.org/S12-1033 \# Emotional Tweets . In * SEM 2012: The First Joint Conference on Lexical and Computational Semantics -- Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation ( S em E val 2012) , pages 246--255, Montr \'...

    work page 2012

  17. [17]

    Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. https://doi.org/10.18653/v1/S16-1003 S em E val-2016 Task 6: Detecting Stance in Tweets . In Proceedings of the 10th International Workshop on Semantic Evaluation ( S em E val-2016) , pages 31--41, San Diego, California. Association for Computational Linguistics

    work page doi:10.18653/v1/s16-1003 2016

  18. [18]

    Mohammad and Svetlana Kiritchenko

    Saif M. Mohammad and Svetlana Kiritchenko. 2015. https://doi.org/10.1111/coin.12024 Using Hashtags to Capture Fine Emotion Categories from Tweets . Computational Intelligence, 31:301 -- 326

    work page doi:10.1111/coin.12024 2015

  19. [19]

    Mohammad and Peter D

    Saif M. Mohammad and Peter D. Turney. 2013. https://doi.org/10.1111/j.1467-8640.2012.00460.x Crowdsourcing a Word-Emotion Association Lexicon . Computational Intelligence, 29

    work page doi:10.1111/j.1467-8640.2012.00460.x 2013

  20. [20]

    Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020. https://doi.org/10.18653/v1/2020.emnlp-demos.2 BERTweet : A pre-trained language model for English Tweets . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pages 9--14. Association for Computational Linguistics

    work page doi:10.18653/v1/2020.emnlp-demos.2 2020

  21. [21]

    Robert Plutchik. 2001. http://www.jstor.org/stable/27857503 The Nature of Emotions . American Scientist, 89(4):344--350

    work page arXiv 2001

  22. [22]

    Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2020. https://doi.org/10.1609/aaai.v34i01.5385 Automatically Neutralizing Subjective Bias in Text . Proceedings of the AAAI Conference on Artificial Intelligence, 34(01):480--489

    work page doi:10.1609/aaai.v34i01.5385 2020

  23. [23]

    Tim Sainburg, Leland McInnes, and Timothy Q Gentner. 2021. https://doi.org/10.1162/neco_a_01434 Parametric UMAP Embeddings for Representation and Semisupervised Learning . Neural Computation, 33(11):2881--2907

    work page doi:10.1162/neco_a_01434 2021

  24. [24]

    Smith, and Yejin Choi

    Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi. 2020. https://doi.org/10.18653/v1/2020.acl-main.486 Social Bias Frames: Reasoning about Social and Power Implications of Language . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 5477--5490, Online. Association for Com...

    work page doi:10.18653/v1/2020.acl-main.486 2020

  25. [25]

    Sherry B Schnake and Janet B Ruscher. 1998. https://doi.org/10.1177/0261927X980174004 Modern Racism as a predictor of the Linguistic Intergroup Bias . Journal of Language and Social Psychology, 17(4):484--491

    work page doi:10.1177/0261927x980174004 1998

  26. [26]

    Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.291 Towards C ontrollable B iases in L anguage G eneration . In Findings of the Association for Computational Linguistics: EMNLP 2020 , pages 3239--3254, Online. Association for Computational Linguistics

    work page doi:10.18653/v1/2020.findings-emnlp.291 2020

  27. [27]

    Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. https://doi.org/10.18653/v1/D19-1339 The Woman Worked as a Babysitter: On Biases in Language Generation . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) ...

    work page doi:10.18653/v1/d19-1339 2019

  28. [28]

    Teun A Van Dijk. 2009. https://doi.org/10.1017/CBO9780511575273 Society and Discourse: How Social Contexts Influence Text and Talk . Cambridge University Press

    work page doi:10.1017/cbo9780511575273 2009

  29. [29]

    Sida Wang and Christopher Manning. 2012. https://aclanthology.org/P12-2018 Baselines and Bigrams: Simple, Good Sentiment and Topic Classification . In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages 90--94, Jeju Island, Korea. Association for Computational Linguistics

    work page 2012

  30. [30]

    Big Data

    Wenbo Wang, Lu Chen, Krishnaprasad Thirunarayan, and Amit P. Sheth. 2012. https://doi.org/10.1109/SocialCom-PASSAT.2012.119 Harnessing Twitter "Big Data" for Automatic Emotion Identification . In 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing, pages 587--592

    work page doi:10.1109/socialcom-passat.2012.119 2012

  31. [31]

    Albert Webson, Zhizhong Chen, Carsten Eickhoff, and Ellie Pavlick. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.335 Are `` Undocumented Workers '' the Same as `` Illegal Aliens '' ? D isentangling Denotation and Connotation in Vector Spaces . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 409...

    work page doi:10.18653/v1/2020.emnlp-main.335 2020

  32. [32]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. https://www.aclweb.org/a...

    work page 2020

  33. [33]

    Samira Zad, Joshuan Jimenez, and Mark Finlayson. 2021. https://doi.org/10.18653/v1/2021.woah-1.11 Hell Hath No Fury? Correcting Bias in the NRC Emotion Lexicon . In Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021) , pages 102--113, Online. Association for Computational Linguistics

    work page doi:10.18653/v1/2021.woah-1.11 2021