Pith. sign in

REVIEW 4 major objections 5 minor 64 references

RLHF preference labels may encode annotator emotional state as well as response quality, and the resulting bias can survive aggregation, reward modeling, and policy optimization.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 16:18 UTC pith:43QQJCZ7

load-bearing objection A credible hypothesis-and-audit paper with a sound conditional pipeline analysis, but the central empirical premise—that RLHF preference raters are strained and shift toward the SLEA pattern—is unverified, and the proposed audit cannot yet test it. the 4 major comments →

arxiv 2607.16195 v2 pith:43QQJCZ7 submitted 2026-04-14 cs.AI

Rater State Bias in RLHF Preference Data: An Audit Framework

Elena Kopteva , Vitaliy Hlynianyi-Zhuk This is my paper
classification cs.AI
keywords rater state shiftRLHFpreference data biasannotator statesurvival level emotional authenticityreward model overoptimizationbias amplificationalignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the pairwise preference labels used to train reward models in RLHF are not just noisy measurements of response quality: they can also carry a systematic trace of the annotator's psychological state at the moment of judgment. If annotation work places raters under sustained stress, their preferences may tilt toward responses that provide immediate emotional validation, affect mirroring, and prolonged emotional contact before redirection—a pattern the paper names survival level emotional authenticity (SLEA). The critical move is that this shift is correlated across raters who share working conditions, so it does not cancel under aggregation. The paper shows analytically how such a shift can enter the preference signal, be absorbed by a reward model trained with a logistic pairwise-comparison loss, and then be amplified when a policy optimizes against that reward. Its contribution is a concrete audit framework with five falsifiable predictions, executable on publicly available models, that could detect this bias from the outside without proprietary rater data.

Core claim

The paper's central claim is that a rater state shift—a systematic, sustained change in an annotator's affect, attention, or regulation under annotation conditions—can become a rater state confound in preference labels, and when shared across raters, a correlated rater state bias in the learned reward function. Concretely, if a fraction f of annotations are produced under a shift delta(x) on prompt class x, the aggregate preference probability becomes p0 + f*delta(x); correlation does not change this mean shift but does reduce the effective number of independent observations, letting the bias pass agreement-based quality control. A reward model fit to the shifted preferences absorbs the shif

What carries the argument

The load-bearing mechanism is the preference-shift model p_shift = p0 + delta(x): a per-prompt change in the probability that raters prefer the SLEA-style response, combined with a clustering argument showing that correlation across raters shrinks the effective sample size (design effect) and so keeps the aggregate shift alive through quality filtering. The observable signature is survival level emotional authenticity (SLEA), a composite of lexical features (density of unconditional validation phrases), pragmatic features (validation-to-redirection and validation-to-question ratios), discourse features (affect mirroring, turn-initial acknowledgment, clinical framing), and safety-boundary fea

Load-bearing premise

The load-bearing premise is empirical: real RLHF preference annotators must actually work under sustained psychological strain, and that strain must shift their pairwise choices toward immediate validation, affect mirroring, and delayed redirection (the SLEA pattern)—the paper's own Section 5 notes this remains unestablished for preference-ranking workflows.

What would settle it

A rater-level exposure study: collect timestamped annotation logs with cumulative sensitive-content exposure and prompt topic, then test whether preference for unconditionally validating responses rises monotonically with exposure duration. If it does not, after controlling for topic and model version, the central mechanism has no effect to propagate.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Held-out agreement with the same rater population will not reveal the confound; reward-model audits should add prompt-category-dependent emotional-profile measurements.
  • Models trained with different rater populations and conditions should diverge most on trauma-adjacent prompts and converge on neutral prompts, so cross-model divergence on affect-laden prompts becomes a fingerprint.
  • If the bias is real, affected models should show high validation-to-redirection ratios, near-zero validation onset, and lower refusal rates on ambiguous safety prompts—on distress categories only, not as a uniform warm tone.
  • Annotation workforce conditions (pay, content exposure, support) become a data-quality variable: improving them would change the preference signal itself, not just worker wellbeing.
  • The alignment target is called into question: whose preferences, in what state, define alignment becomes a technical question, and learned emotional style may relate to downstream loneliness and psychosocial outcomes.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • By the paper's own correlation logic, preference datasets should report cluster-level statistics—intraclass correlation and effective independent sample size—alongside inter-annotator agreement; without them, quality-control claims are underdetermined.
  • The audit logic can be inverted into a stress test for deployed assistants: if a model shows SLEA-pattern behavior on distress prompts, that is not proof of rater strain, but it is a concrete reason to inspect the provenance of its preference data.
  • A natural extension the paper leaves implicit: apply the pilot to open-weight models whose RLHF data provenance is documented (different vendors and geographies) rather than only to models sharing one base architecture, which would isolate rater-population effects from alignment-method effects.
  • If SLEA bias is entrenched, system-prompt overlays may suppress its visible expression while leaving the underlying reward-model distortion intact; the ambiguous-safety-prompt endpoint may be the most sensitive probe.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a structured confound in RLHF preference data: rater state shift—systematic changes in a rater's affective and regulatory state under annotation conditions—can enter pairwise preference labels, and when shared across raters, can survive aggregation, be absorbed by reward-model training, and be amplified by policy optimization. It formalizes this with a mean-field model (Eqs. 4–5), a Kish design-effect analysis of correlated raters (Eq. 6), a Bradley-Terry absorption expression (Eq. 7), and overoptimization equations (Eqs. 8–9). The paper then introduces 'survival level emotional authenticity' (SLEA) as a candidate output signature, defines lexical/pragmatic/discourse/safety features, states five falsifiable predictions (P1–P5), and presents an audit protocol and pilot plan using public instruction-tuned models. The paper explicitly does not claim to identify the training history of any specific deployed model, and Section 5 states that direct evidence for rater strain in preference-ranking workflows is open.

Significance. If validated, the framework would extend annotator-bias research from stable rater traits to within-rater state variation, and it would provide an external audit tool for a potential source of structured bias in RLHF. The formal conditional analysis is a genuine strength: it correctly shows that a systematic preference shift need not cancel under averaging, and that correlation reduces effective sample size, making detection harder. The detailed, preregistered-style measurement protocol and the explicit separation of instrument validation from causal testing are also valuable. However, the significance is currently prospective: the load-bearing empirical premises—that preference raters are strained and that strain shifts preferences toward SLEA—are unverified, and the proposed pilot cannot test them. The paper fits no parameters; f, δ, and ρ are illustrative sensitivity values, which is appropriate for a framework paper but limits quantitative claims. The paper is best read as a hypothesis-generation and audit-design contribution rather than an empirical demonstration.

major comments (4)
  1. [§5 and §6] The central claim that rater state shift can contribute a measurable component to the learned preference signal rests on two premises the paper does not establish: (i) RLHF preference-ranking raters work under conditions of sustained psychological strain, and (ii) strain biases comparisons toward the SLEA direction. Section 5 states 'What remains open is whether comparable conditions characterized specific RLHF preference ranking workflows,' and the cited evidence is from content moderation and safety labeling, not preference ranking. All propagation results (Eqs. 4–7) are conditional on a nonzero systematic δ(x). The audit protocol (§7) studies model outputs, not rater states, so a positive audit cannot confirm the premise. The authors should either supply direct rater-level or experimental evidence, or explicitly scope the central claim as a conditional hypothesis and state what eviden
  2. [§6 and Eq. (4)] The formal framework is sign-agnostic: δ(x) in Eq. (4) may be positive or negative, and the propagation equations hold either way. The empirical content of the SLEA hypothesis is that δ(x)>0 for immediate validation, affect mirroring, and delayed redirection on distress prompts. The mechanistic grounding (attachment theory, feelings-as-information) is analogical and does not directly address the rater's task: raters compare third-party responses to someone else's distress, rather than receiving support themselves. A stressed rater could plausibly prefer concise redirection or become impatient with emotional content, yielding δ(x)<0. The paper should provide empirical evidence for the predicted sign, or design the predictions as two-sided tests with explicit alternative hypotheses; otherwise a null SLEA result would not test the propagation framework.
  3. [§7.3 and §6] The pilot study is explicitly an instrument validation: 'It does not test whether rater state caused any observed differences.' A positive result—category-specific model divergence—is consistent with SLEA but also with the alternatives listed in §6 (intentional design, architectural differences, user projection, post-training modifications). The prompt-category×model interaction reduces the space of alternatives but does not isolate rater state, because any factor that affects distress prompts unevenly across models would produce the same interaction. To strengthen the claim that the audit detects rater state bias rather than generic emotional style, the paper should specify a pattern of results that would uniquely implicate rater state (e.g., P1's exposure-affect gradient, or within-rater drift correlated with exposure proxies) and state thresholds that distinguish it from the alternati
  4. [§4.1 and Eq. (6)] The design-effect argument is used to support the claim that correlated rater state bias can survive aggregation, but the mean shift fδ in Eq. (5) is invariant to ρ; correlation changes the variance of the aggregate estimate, not its expectation. A systematic shift shared by independent raters would also survive aggregation and enter reward-model training. The paper should clarify that correlation is not necessary for the mean shift to persist—it affects detectability and the effective sample size (Neff in Table 1)—and that 'correlated' refers to a property affecting estimate variance, not the mechanism that prevents cancellation.
minor comments (5)
  1. [§3.3] The exposure-proxy discussion would benefit from a table linking each proxy (sensitive-prompt share, session length, timestamps, topic clusters) to the specific prediction it tests and the data source required.
  2. [§4.2, Eq. (7)] The shift Δr is derived under the assumption that the fitted reward model exactly matches the aggregate Bradley-Terry probability. Finite-sample noise, model capacity, and regularization will typically attenuate or alter this shift; please present it explicitly as a first-order approximation.
  3. [§6, P3] The footnote correctly says that public user discourse is 'consistent with P3 but does not by itself confirm the mechanism.' Consider moving this caveat into the main text, since the prediction list as printed could be read as if public discourse were evidence for the mechanism.
  4. [§7.1, UVPD] The definition of UVPD uses a seed lexicon and a cosine-similarity threshold of 0.85. The model-minus-human difference may be sensitive to this threshold and to the choice of seed corpus; report sensitivity analyses in the pilot.
  5. [General] Minor copyedit: for example, 'Under annotation conditions, we mean...' in §1 is awkwardly phrased, and the abstract repeats 'rater state shift' several times. These are presentation issues only.

Circularity Check

0 steps flagged

No circularity: the propagation argument is conditional and parameter values are illustrative; SLEA is hypothesized from external literature, not fitted.

full rationale

The paper contains no circular derivation. Section 4's formal chain is explicitly conditional: Eq. (4) defines δ(x) as the difference between the shifted and baseline preference probabilities, and Eqs. (5) and (7) are algebraic consequences of that definition together with the Bradley-Terry model. The parameters f, δ, and ρ are presented as illustrative sensitivity values ('the parameter values below are illustrative and require empirical validation'), not fitted to any outcome the paper claims to predict. The SLEA response pattern is hypothesized from external psychological literature (attachment theory, feelings-as-information) rather than derived from the preference data the audit would explain, and the paper explicitly states the feature reliability and validity 'remain to be demonstrated.' The empirical premise is candidly limited: 'What remains open is whether comparable conditions characterized specific RLHF preference ranking workflows,' and the pilot is framed as instrument validation that 'does not test whether rater state caused any observed variation.' There are no load-bearing self-citations; all cited formal and empirical results are external prior work. The identified weakness—that the central mechanism requires unverified premises about rater strain and its directional effect on preferences—is a correctness or external-validity risk, not a circularity. The audit predictions are designed as falsifiable discriminators with pre-registered success criteria, not as restatements of the inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The paper's formal contribution rests on a small set of illustrative free parameters (f, δ, ρ, n) that are explicitly not fitted, plus domain assumptions about annotator clustering and the psychological direction of the shift. The central empirical premise—that RLHF preference raters experience sustained strain that shifts preferences toward SLEA—is itself a hypothesis, not a measured input. The SLEA construct is the main invented entity, with no independent validation to date.

free parameters (5)
  • f (fraction of annotations under rater state shift) = 0.10 and 0.20 (illustrative)
    Used in Eq. 5 and Table 1 to illustrate the mean shift f·δ; explicitly not fitted to data.
  • δ(x) (per-prompt preference shift) = 0.10 and 0.15 (illustrative)
    Per-prompt-class shift in preference probability; Table 1 uses these illustrative values.
  • ρ (intraclass correlation coefficient) = 0.0, 0.1, 0.2 (illustrative)
    ICC among annotation workers in a cluster; cited to survey literature but not measured on RLHF raters.
  • n (cluster size) = 200 (illustrative)
    Assumed cluster size in the Kish design effect (Eq. 6); used to compute Neff in Table 1.
  • SLEA feature and pilot thresholds = cosine similarity >0.85; Cohen's κ>0.6; Pearson r>0.7; |d|>0.3; B=10,000
    Hand-chosen operational thresholds in Section 7; no empirical calibration.
axioms (5)
  • standard math RLHF preference labels follow the Bradley-Terry model (Eq. 1).
    Invoked in Section 3.1 and used throughout Section 4; the model is standard in the RLHF literature [36, 1].
  • domain assumption A stable target preference probability exists under the null hypothesis.
    Section 3.2 contrasts this stable-target picture with state-dependent preferences; the null is the standard RLHF abstraction.
  • ad hoc to paper Rater state shift acts as an additive shift in preference probability: p_shift = p0 + δ(x).
    Eq. 4 introduces this first-order mean-field approximation; the paper notes it ignores higher-order dependence among annotators.
  • ad hoc to paper Sustained psychological strain shifts raters' preferences toward immediate validation, affect mirroring, and delayed redirection (the SLEA direction).
    Section 6 proposes this direction based on attachment theory and feelings-as-information; it is not directly evidenced in RLHF preference data.
  • domain assumption RLHF annotation workforces are clustered by vendor, region, and shared conditions, producing correlated exposures.
    Section 3.4 and Section 5 rely on public reporting about content-moderation labor structures [37–39]; the paper acknowledges these reports do not show overlap with RLHF preference raters.
invented entities (2)
  • Survival level emotional authenticity (SLEA) no independent evidence
    purpose: Candidate output signature of rater state bias: high unconditional validation, low distancing, high emotional mirroring, delayed redirection on distress prompts.
    The abstract and Section 7 state that the feature construct's reliability and validity 'remain to be demonstrated'; it has no external falsifiable handle beyond the paper's own proposed audit.
  • Rater state shift / rater state confound / correlated rater state bias no independent evidence
    purpose: Latent constructs connecting annotation conditions to preference labels and learned reward.
    Defined conceptually in Section 1; not directly observable in public RLHF data and only proxied by timestamps, topic clusters, and shared vendor structure.

pith-pipeline@v1.3.0-alltime-deepseek · 19063 in / 11828 out tokens · 102719 ms · 2026-08-02T16:18:39.970227+00:00 · methodology

0 comments
read the original abstract

We identify a structured confound in Reinforcement Learning from Human Feedback (RLHF). Pairwise preference labels are intended to reflect the compared outputs, but they may also reflect the rater's state during annotation. Under sustained stressful or distressing conditions, raters' preferences may shift over time, so that preference data encode rater state alongside judgments about response quality. We argue that, if present, such shifts would differ from ordinary disagreement or random label noise. They would be state dependent, could be shared across annotators under similar conditions, and would not necessarily cancel during aggregation, reward modeling, and policy optimization. We propose rater state shift as a plausible and testable source of structured bias in RLHF preference data. This paper develops a hypothesis and an audit framework for studying this source of bias. We define rater state shift, rater state confound, and correlated rater state bias. We also propose survival level emotional authenticity as a candidate output signature, defined by lexical, pragmatic, discourse, and safety features whose reliability and validity remain to be demonstrated. We analyze the conditions under which correlated rater state bias would not be averaged out during aggregation and could enter the learned reward signal. We state five predictions that distinguish this mechanism from generic engagement optimization, together with effect size thresholds for an initial audit, and note which require proprietary data. Finally, we present an audit protocol and pilot study plan that can be applied to publicly available instruction tuned models. We do not infer the training history of any specific deployed model.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

64 extracted references · 5 canonical work pages

  1. [1]

    Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei

    Paul F. Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. InAdvances in Neural Information Processing Systems 30 (NeurIPS 2017), 2017. URL https://papers.nips.cc/paper files/paper /2017/hash/d5e2c0adad503c91f91df240d0cd4e49-Abstract.html

    2017

  2. [2]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. InAdvances in Neural Information Processing Systems 35 (NeurIPS 2022), pages 27730–27744, 2022. URL https://papers.nips.cc/paper fil es/pa...

    2022

  3. [3]

    Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. Annotators with attitudes: How annotator beliefs and identities bias toxic language detection. InProceedings of the 2022 Conference of the North American Chapter of the As- sociation for Computational Linguistics: Human Language Technologies, pages 5884–5906. Associa...

    2022

  4. [4]

    NLPo- sitionality: Characterizing design biases of datasets and models

    Sebastin Santy, Jenny Liang, Ronan Le Bras, Katharina Reinecke, and Maarten Sap. NLPo- sitionality: Characterizing design biases of datasets and models. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 20 pages 9080–9102, Toronto, Canada, 2023. Association for Computational Linguistics. URL...

    2023

  5. [5]

    Gordon, Michelle S

    Mitchell L. Gordon, Michelle S. Lam, Joon Sung Park, Kayur Patel, Jeff Hancock, Tatsunori Hashimoto, and Michael S. Bernstein. Jury learning: Integrating dissenting voices into machine learning models. InCHI Conference on Human Factors in Computing Systems, pages 1–19,

  6. [6]

    Truth is a lie: Crowd truth and the seven myths of human annotation.AI Magazine, 36(1):15–24, 2015

    Lora Aroyo and Chris Welty. Truth is a lie: Crowd truth and the seven myths of human annotation.AI Magazine, 36(1):15–24, 2015

    2015

  7. [7]

    The “problem” of human label variation: On ground truth in data, modeling and evaluation

    Barbara Plank. The “problem” of human label variation: On ground truth in data, modeling and evaluation. InProceedings of the 2022 Conference on Empirical Methods in Natural Lan- guage Processing, pages 10671–10682. Association for Computational Linguistics, 2022. URL https://aclanthology.org/2022.emnlp-main.730/

    2022

  8. [8]

    Dealing with dis- agreements: Looking beyond the majority vote in subjective annotations.Transactions of the Association for Computational Linguistics, 10:92–110, 2022

    Aida Mostafazadeh Davani, Mark D ´ ıaz, and Vinodkumar Prabhakaran. Dealing with dis- agreements: Looking beyond the majority vote in subjective annotations.Transactions of the Association for Computational Linguistics, 10:92–110, 2022. URL https://direct.mit.edu/tacl/ article/doi/10.1162/tacl a 00444/109274/Dealing-with-Disagreements-Looking-Beyond-the

    doi:10.1162/tacl 2022

  9. [9]

    Gray and Siddharth Suri.Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass

    Mary L. Gray and Siddharth Suri.Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Harper Business, New York, 2019

    2019

  10. [10]

    Roberts.Behind the Screen: Content Moderation in the Shadows of Social Media

    Sarah T. Roberts.Behind the Screen: Content Moderation in the Shadows of Social Media. Yale University Press, New Haven, 2019

    2019

  11. [11]

    Incorporating worker perspectives into MTurk annotation practices for NLP

    Olivia Huang, Eve Fleisig, and Dan Klein. Incorporating worker perspectives into MTurk annotation practices for NLP. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1010–1028, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.64. URL https://aclantholo gy.o...

    work page doi:10.18653/v1/2023.emnlp-main.64 2023

  12. [12]

    Analyzing dataset anno- tation quality management in the wild.Computational Linguistics, 50(3):817–866, September

    Jan-Christoph Klie, Richard Eckart de Castilho, and Iryna Gurevych. Analyzing dataset anno- tation quality management in the wild.Computational Linguistics, 50(3):817–866, September

  13. [13]

    Discovering language model behaviors with model-written evaluations.arXiv preprint arXiv:2212.09251, 2022

    Ethan Perez, Sam Ringer, Kamil˙ e Lukoˇ si¯ ut˙ e, et al. Discovering language model behaviors with model-written evaluations.arXiv preprint arXiv:2212.09251, 2022

    Pith/arXiv arXiv 2022

  14. [14]

    Bowman, et al

    Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, et al. Towards understanding sycophancy in language models.arXiv preprint arXiv:2310.13548, 2023

    Pith/arXiv arXiv 2023

  15. [15]

    Procaccia

    Itai Shapira, Gerdus Benad` e, and Ariel D. Procaccia. How rlhf amplifies sycophancy.arXiv preprint arXiv:2602.01002, 2026

    arXiv 2026

  16. [16]

    A survey on intersectional fairness in machine learning: Notions, mitigation, and challenges

    Usman Gohar and Lu Cheng. A survey on intersectional fairness in machine learning: Notions, mitigation, and challenges. InProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, pages 6619–6627, 2023. URL https://www.ijcai.org/proceedings/20 23/742. Survey Track. 21

    2023

  17. [17]

    Scaling laws for reward model overopti- mization

    Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overopti- mization. InProceedings of the 40th International Conference on Machine Learning, vol- ume 202 ofProceedings of Machine Learning Research, pages 10835–10866, 2023. URL https://proceedings.mlr.press/v202/gao23h.html

    2023

  18. [18]

    Bradley Knox, Chelsea Finn, and Scott Niekum

    Rafael Rafailov, Yaswanth Chittepu, Ryan Park, Harshit Sikchi, Joey Hejna, W. Bradley Knox, Chelsea Finn, and Scott Niekum. Scaling laws for reward model overoptimization in direct alignment algorithms, 2024. URL https://arxiv.org/abs/2406.02900

    Pith/arXiv arXiv 2024

  19. [19]

    Men also like shopping: Reducing gender bias amplification using corpus-level constraints

    Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979–2989. Association for Computational Linguistics, 2017. URL https://aclanthology.org/D17-1323/

    2017

  20. [20]

    Directional bias amplification

    Angelina Wang and Olga Russakovsky. Directional bias amplification. InProceedings of the 38th International Conference on Machine Learning, volume 139 ofProceedings of Machine Learning Research, pages 10882–10893, 2021. URL https://proceedings.mlr.press/v139/wan g21t.html

    2021

  21. [21]

    A systematic study of bias amplification, 2022

    Melissa Hall, Laurens van der Maaten, Laura Gustafson, and Aaron Adcock. A systematic study of bias amplification, 2022. URL https://arxiv.org/abs/2201.11706

    Pith/arXiv arXiv 2022

  22. [22]

    Bell, Levent Sagun, and Elvis Dohmatob

    Arjun Subramonian, Samuel J. Bell, Levent Sagun, and Elvis Dohmatob. An effective theory of bias amplification, 2024. URL https://arxiv.org/abs/2410.17263

    Pith/arXiv arXiv 2024

  23. [23]

    Pennebaker, Ryan L

    James W. Pennebaker, Ryan L. Boyd, Kayla Jordan, and Kate Blackburn. The development and psychometric properties of LIWC2015. Technical report, University of Texas at Austin, 2015

    2015

  24. [24]

    C. J. Hutto and Eric Gilbert. V ADER: A parsimonious rule-based model for sentiment analysis of social media text. InProceedings of the International AAAI Conference on Web and Social Media, volume 8, pages 216–225, 2014. doi: 10.1609/icwsm.v8i1.14550

    doi:10.1609/icwsm.v8i1.14550 2014

  25. [25]

    Mohammad and Peter D

    Saif M. Mohammad and Peter D. Turney. Crowdsourcing a word-emotion association lexicon. Computational Intelligence, 29(3):436–465, 2013

    2013

  26. [26]

    GoEmotions: A dataset of fine-grained emotions

    Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. GoEmotions: A dataset of fine-grained emotions. InProceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 4040–4054. Association for Computational Linguistics, 2020. URL https://aclanthology.org/2020.acl-main.372/

    2020

  27. [27]

    Towards empa- thetic open-domain conversation models: A new benchmark and dataset

    Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. Towards empa- thetic open-domain conversation models: A new benchmark and dataset. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5370–5381. Association for Computational Linguistics, 2019. URL https://aclanthology.org/P19-1534/

    2019

  28. [28]

    Basic Books, 1969

    John Bowlby.Attachment and Loss: Volume I: Attachment. Basic Books, 1969

    1969

  29. [29]

    Shaver, and Daphna Pereg

    Mario Mikulincer, Phillip R. Shaver, and Daphna Pereg. Attachment theory and affect regula- tion: The dynamics, development, and cognitive consequences of attachment-related strategies. Motivation and Emotion, 27(2):77–102, 2003. doi: 10.1023/A:1024515519160. 22

    work page doi:10.1023/a:1024515519160 2003

  30. [30]

    Theodore E. A. Waters and Harriet S. Waters. Measuring attachment representations. In Jude Cassidy and Phillip R. Shaver, editors,Handbook of Attachment: Theory, Research, and Clinical Applications, pages 235–260. Guilford Press, 3 edition, 2019

    2019

  31. [31]

    Mary D. S. Ainsworth, Mary C. Blehar, Everett Waters, and Sally N. Wall.Patterns of Attachment: A Psychological Study of the Strange Situation. Lawrence Erlbaum Associates, 1978

    1978

  32. [32]

    Carl R. Rogers. The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting Psychology, 21(2):95–103, 1957. doi: 10.1037/h0045357

    doi:10.1037/h0045357 1957

  33. [33]

    Truax and Robert R

    Charles B. Truax and Robert R. Carkhuff.Toward Effective Counseling and Psychotherapy: Training and Practice. Aldine, 1967

    1967

  34. [34]

    Stern.The Interpersonal World of the Infant

    Daniel N. Stern.The Interpersonal World of the Infant. Basic Books, 1985

    1985

  35. [35]

    Feeney and Nancy L

    Brooke C. Feeney and Nancy L. Collins. Interpersonal safe haven and secure base caregiving processes in adulthood. In W. Steven Rholes and Jeffry A. Simpson, editors,Adult Attachment: Theory, Research, and Clinical Implications, pages 300–338. Guilford Press, 2004

    2004

  36. [36]

    Ralph Allan Bradley and Milton E. Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons.Biometrika, 39(3/4):324–345, 1952. doi: 10.2307/2334029

    doi:10.2307/2334029 1952

  37. [37]

    Exclusive: OpenAI used kenyan workers on less than$2 per hour to make ChatGPT less toxic, January 2023

    Billy Perrigo. Exclusive: OpenAI used kenyan workers on less than$2 per hour to make ChatGPT less toxic, January 2023. URL https://time.com/6247678/openai-chatgpt-kenya-w orkers/

    arXiv 2023

  38. [38]

    Kenyan workers with AI jobs thought they had tickets to the future until the grim reality set in, November 2024

    Lesley Stahl, Aliza Chasan, Shachar Bar-On, and Jinsol Jung. Kenyan workers with AI jobs thought they had tickets to the future until the grim reality set in, November 2024. URL https://www.cbsnews.com/news/ai-work-kenya-exploitation-60-minutes/. Accessed 2026-04- 13

    2024

  39. [39]

    SF tech startup Scale AI, worth$13.8b, accused of widespread wage theft, December 2024

    Stephen Council. SF tech startup Scale AI, worth$13.8b, accused of widespread wage theft, December 2024. URL https://www.sfgate.com/tech/article/sf-tech-startup-scale-ai-sued-w age-theft-19976761.php. Accessed 2026-04-13

    2024

  40. [40]

    What is an intracluster correlation coef- ficient? crucial concepts for primary care researchers.The Annals of Family Medicine, 2(3): 204–208, 2004

    Shersten Killip, Ziyad Mahfoud, and Kevin Pearce. What is an intracluster correlation coef- ficient? crucial concepts for primary care researchers.The Annals of Family Medicine, 2(3): 204–208, 2004. doi: 10.1370/afm.141

    doi:10.1370/afm.141 2004

  41. [41]

    John Wiley & Sons, New York, 1965

    Leslie Kish.Survey Sampling. John Wiley & Sons, New York, 1965

    1965

  42. [42]

    Thompson, Douglas H

    David M. Thompson, Douglas H. Fernald, and James W. Mold. Intraclass correlation coef- ficients typical of cluster-randomized studies: Estimates from the robert wood johnson pre- scription for health projects.The Annals of Family Medicine, 10(3):235–240, 2012. doi: 10.1370/afm.1347

    work page doi:10.1370/afm.1347 2012

  43. [43]

    R. E. Davis, M. P. Couper, N. K. Janz, C. H. Caldwell, and K. Resnicow. Interviewer effects in public health surveys.Health Education Research, 25(1):14–26, 2010. doi: 10.1093/her/cyp046

    work page doi:10.1093/her/cyp046 2010

  44. [44]

    Deep learning is robust to massive label noise, 2017

    David Rolnick, Andreas Veit, Serge Belongie, and Nir Shavit. Deep learning is robust to massive label noise, 2017. URL https://arxiv.org/abs/1705.10694. 23

    Pith/arXiv arXiv 2017

  45. [45]

    Dhillon, Pradeep K

    Nagarajan Natarajan, Inderjit S. Dhillon, Pradeep K. Ravikumar, and Ambuj Tewari. Learning with noisy labels. InAdvances in Neural Information Processing Systems 26 (NeurIPS 2013),

    2013

  46. [46]

    Learning from noisy labels with deep neural networks: A survey.IEEE Transactions on Neural Networks and Learning Systems, 34(11):8135–8153, 2023

    Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: A survey.IEEE Transactions on Neural Networks and Learning Systems, 34(11):8135–8153, 2023

    2023

  47. [47]

    Bharucha, Sukrit Venkatagiri, Martin J

    Miriah Steiger, Timir J. Bharucha, Sukrit Venkatagiri, Martin J. Riedl, and Matthew Lease. The psychological well-being of content moderators: The emotional labor of commercial moder- ation and avenues for improving support. InCHI Conference on Human Factors in Computing Systems, pages 1–14, 2021. doi: 10.1145/3411764.3445092

    arXiv 2021

  48. [48]

    ‘it’s destroyed me completely’: Kenyan moderators decry toll of training of AI models, August 2023

    Richard Nieva. ‘it’s destroyed me completely’: Kenyan moderators decry toll of training of AI models, August 2023. URL https://www.theguardian.com/technology/2023/aug/02/kenya n-workers-for-openai-facebook-say-they-paid-heavy-price-for-training-ai-models

    2023

  49. [49]

    Kim, David K

    Heejung S. Kim, David K. Sherman, and Shelley E. Taylor. Culture and social support. American Psychologist, 63(6):518–526, 2008

    2008

  50. [50]

    Jozefien De Leersnyder, Batja Mesquita, and Heejung S. Kim. Where do my emotions belong? a study of immigrants’ emotional acculturation.Personality and Social Psychology Bulletin, 37(4):451–463, 2011. doi: 10.1177/0146167211399103

    work page doi:10.1177/0146167211399103 2011

  51. [51]

    On the universality and cultural specificity of emotion recognition: A meta-analysis.Psychological Bulletin, 128(2):203–235, 2002

    Hillary Anger Elfenbein and Nalini Ambady. On the universality and cultural specificity of emotion recognition: A meta-analysis.Psychological Bulletin, 128(2):203–235, 2002

    2002

  52. [52]

    Heine, and Ara Norenzayan

    Joseph Henrich, Steven J. Heine, and Ara Norenzayan. The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3):61–83, 2010

    2010

  53. [53]

    The GPT-4o shock: Emotional attachment to AI models and its impact on regulatory acceptance: A cross-cultural analysis of the immediate transition from GPT-4o to GPT-5, 2025

    Hiroki Naito. The GPT-4o shock: Emotional attachment to AI models and its impact on regulatory acceptance: A cross-cultural analysis of the immediate transition from GPT-4o to GPT-5, 2025. URL https://arxiv.org/abs/2508.16624. Preprint

    arXiv 2025

  54. [54]

    The backlash over OpenAI’s decision to retire GPT-4o shows how danger- ous AI companions can be, February 2026

    Amanda Silberling. The backlash over OpenAI’s decision to retire GPT-4o shows how danger- ous AI companions can be, February 2026. URL https://techcrunch.com/2026/02/06/the-b acklash-over-openais-decision-to-retire-gpt-4o-shows-how-dangerous-ai-companions-can-be/. Accessed 2026-04-13

    2026

  55. [55]

    Barsalou

    Lawrence W. Barsalou. Grounded cognition.Annual Review of Psychology, 59:617–645, 2008

    2008

  56. [56]

    Feelings-as-information theory

    Norbert Schwarz. Feelings-as-information theory. InHandbook of Theories of Social Psychol- ogy, volume 1, pages 289–308. SAGE Publications, 2011

    2011

  57. [57]

    When human-AI interactions become parasocial

    Takuya Maeda and Anabel Quan-Haase. When human-AI interactions become parasocial. InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1914–1928, 2024. doi: 10.1145/3630106.3659003

    arXiv 2024

  58. [58]

    Continuum, 2005

    Ken Hyland.Metadiscourse: Exploring Interaction in Writing. Continuum, 2005. 24

    2005

  59. [59]

    Dailydialog: A manually labelled multi-turn dialogue dataset

    Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. Dailydialog: A manually labelled multi-turn dialogue dataset. InProceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 986–995. Asian Federation of Natural Language Processing, 2017. URL https://aclanthology.org/I17-1099/

    2017

  60. [60]

    How AI and human behaviors shape psychosocial effects of chatbot use: A longitudinal randomized controlled study, 2025

    Cathy Mengying Fang, Pat Pataranutaporn, Michael Lampe, Jason Phang, Sandhini Agar- wal, Nava Gileadi, Xiaomeng Wang, and Pattie Maes. How AI and human behaviors shape psychosocial effects of chatbot use: A longitudinal randomized controlled study, 2025. URL https://arxiv.org/abs/2503.17473

    Pith/arXiv arXiv 2025

  61. [61]

    Investigating affective use and emotional well-being on ChatGPT, 2025

    Jason Phang, Michael Lampe, Sandhini Agarwal, Cathy Mengying Fang, Pat Pataranutaporn, and Pattie Maes. Investigating affective use and emotional well-being on ChatGPT, 2025. URL https://arxiv.org/abs/2504.03888. 25

    Pith/arXiv arXiv 2025

  62. [2013]

    URL https://proceedings.neurips.cc/paper/2013/hash/3871bd64012152bfb53fdf04b40 1193f-Abstract.html

    2013

  63. [2022]

    doi: 10.1145/3491102.3517444

    arXiv

  64. [2024]

    URL https://aclanthology.org/2024.cl-3.1/

    doi: 10.1162/coli a 00516. URL https://aclanthology.org/2024.cl-3.1/

    doi:10.1162/coli 2024