A framework jointly models annotator-specific NLI labels and explanations using conditioned representations and two explainer architectures, improving predictive performance over baselines.
Toward a perspectivist turn in ground truthing for predictive computing
10 Pith papers cite this work, alongside 59 external citations. Polarity classification is still indexing.
10
Pith papers citing it
59
external citations · OpenAlex
years
2026 10representative citing papers
Ensemble Diversity Optimization jointly learns ensemble weights, size, and a signed diversity regularizer, substantially improving calibration to annotator distributions on subjective text classification.
Demographic-conditioned fusion embeddings improve prediction of perspectivist social meaning interpretations by 5.9-6.5% relative macro PR-AUC over text-only baselines, with ablations confirming demographic signal.
New Ghost Annotator framework uses conformal prediction to show LLMs of different sizes and families produce labels no human annotator chose and align least with 18-30 male Sub-Saharan African annotators across content moderation datasets.
Agreement-based clustering of annotators improves performance on subjective NLP tasks by capturing diverse perspectives better than majority voting or per-annotator modeling.
Large-scale statistical analysis of four harmful language datasets reveals that interactions between annotator characteristics and linguistic cues drive annotation variation, with lexical features and attitudes prominent but patterns varying by dataset.
Analyses of labeled social media sentences and interpretations show 30% divergence in ethos and pathos, greater variability for charged content, and predictive power for audience attitudes toward the author.
Extending language models with annotator-specific layers improves individual moral annotation predictions and reveals perspective variations hidden by label aggregation.
A domain-agnostic framework extracts perspectives from book reviews showing LLMs underrepresent rarer viewpoints relative to human text.
citing papers explorer
-
Fine-Grained Perspectives: Modeling Explanations with Annotator-Specific Rationales
A framework jointly models annotator-specific NLI labels and explanations using conditioned representations and two explainer architectures, improving predictive performance over baselines.
-
Ensemble Diversity Optimization for Subjective Supervision
Ensemble Diversity Optimization jointly learns ensemble weights, size, and a signed diversity regularizer, substantially improving calibration to annotator distributions on subjective text classification.
-
Learning Perspectivist Social Meaning via Demographic-Conditioned Fusion Embeddings
Demographic-conditioned fusion embeddings improve prediction of perspectivist social meaning interpretations by 5.9-6.5% relative macro PR-AUC over text-only baselines, with ablations confirming demographic signal.
-
The Ghost Annotator: a Framework to Explore Human Label Variation in Content Moderation through Conformal Prediction
New Ghost Annotator framework uses conformal prediction to show LLMs of different sizes and families produce labels no human annotator chose and align least with 18-30 male Sub-Saharan African annotators across content moderation datasets.
-
Beyond Majority Voting: Agreement-Based Clustering to Model Annotator Perspectives in Subjective NLP Tasks
Agreement-based clustering of annotators improves performance on subjective NLP tasks by capturing diverse perspectives better than majority voting or per-annotator modeling.
-
Who and What? Using Linguistic Features and Annotator Characteristics to Analyze Annotation Variation
Large-scale statistical analysis of four harmful language datasets reveals that interactions between annotator characteristics and linguistic cues drive annotation variation, with lexical features and attitudes prominent but patterns varying by dataset.
-
How Ethos and Pathos Appeals Resonate in Reader Interpretations of Social Media Messages
Analyses of labeled social media sentences and interpretations show 30% divergence in ethos and pathos, greater variability for charged content, and predictive power for audience attitudes toward the author.
-
Learning Moral Diversity: Modelling Individual Perspectives in Moral Classification of Texts
Extending language models with annotator-specific layers improves individual moral annotation predictions and reveals perspective variations hidden by label aggregation.
-
Evaluating Pluralism in LLMs through Latent Perspectives
A domain-agnostic framework extracts perspectives from book reviews showing LLMs underrepresent rarer viewpoints relative to human text.
- Improving Reproducibility in Evaluation through Multi-Level Annotator Modeling