Evaluating Crowdsourcing Participants in the Absence of Ground-Truth
read the original abstract
Given a supervised/semi-supervised learning scenario where multiple annotators are available, we consider the problem of identification of adversarial or unreliable annotators.
This paper has not been read by Pith yet.
Forward citations
Cited by 3 Pith papers
-
Negative Ontology of True Target for Machine Learning: Towards Evaluation and Learning under Democratic Supervision
By adopting a negative ontology where the true target does not objectively exist, the paper defines Democratic Supervision and derives the EL-MIATTs framework for ML evaluation and learning with Multiple Inaccurate Tr...
-
Negative Ontology of True Target for Machine Learning: Towards Evaluation and Learning under Democratic Supervision
The paper posits that the true target does not exist and introduces the EL-MIATTs framework for evaluation and learning under Democratic Supervision in machine learning.
-
Negative Ontology of True Target for Machine Learning: Towards Evaluation and Learning under Democratic Supervision
The true target does not objectively exist in ML, so models should use multiple inaccurate true targets under democratic supervision via the EL-MIATTs framework for evaluation and learning.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.