Data annotation quality should be assessed with reliability and validity concepts from measurement theory, and annotation problems should be diagnosed by their source (error, ambiguity, impossibility, subjectivity, identity) before intervening.
Irresolvable Disagreement: A Study on Worker Deliberation in Crowd Work.Proceedings of the ACM on Human-Computer Interaction, 2(CSCW): 1–19
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CY 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Data Annotation as Measurement
Data annotation quality should be assessed with reliability and validity concepts from measurement theory, and annotation problems should be diagnosed by their source (error, ambiguity, impossibility, subjectivity, identity) before intervening.