Single-label evaluation hides multi-label recognition ability and explains much of the ImageNetV2 accuracy gap, which shrinks under multi-label-aware metrics and synthetic object-composition tests.
Flaws of ImageNet, Computer Vision's Favourite Dataset
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Since its release, ImageNet-1k dataset has become a gold standard for evaluating model performance. It has served as the foundation for numerous other datasets and training tasks in computer vision. As models have improved in accuracy, issues related to label correctness have become increasingly apparent. In this blog post, we analyze the issues in the ImageNet-1k dataset, including incorrect labels, overlapping or ambiguous class definitions, training-evaluation domain shifts, and image duplicates. The solutions for some problems are straightforward. For others, we hope to start a broader conversation about refining this influential dataset to better serve future research.
citation-role summary
background 1
citation-polarity summary
fields
cs.CV 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
The Impact of the Single-Label Assumption in Image Recognition Benchmarking
Single-label evaluation hides multi-label recognition ability and explains much of the ImageNetV2 accuracy gap, which shrinks under multi-label-aware metrics and synthetic object-composition tests.